princeton-nlp commited on
Commit
300c0d8
1 Parent(s): 3d8fd96

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -11,4 +11,8 @@ license: llama2
11
 
12
  **License**: Must comply with license of Llama2 since it's a model derived from Llama2.
13
 
14
- Sheared-LLaMA-2.7B-Pruned is the model pruned from [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) **without continued pre-training**. We used roughly 0.4B tokens to perform the pruning experiment.
 
 
 
 
 
11
 
12
  **License**: Must comply with license of Llama2 since it's a model derived from Llama2.
13
 
14
+ Sheared-LLaMA-1.3B-Pruned is the model pruned from [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) **without continued pre-training**.
15
+ We used roughly 0.4B tokens to perform the pruning experiment. This model could be a good use to study
16
+ - effective data mixtures for continued pre-training
17
+ - comparisons to other pruning techniques
18
+ - extensive evaluations to understand how pruning affects knowledge and reasoning capabilities of LLMs