Update README.md
Browse files
README.md
CHANGED
@@ -23,12 +23,11 @@ pip install transformers accelerate datasets SentencePiece
|
|
23 |
## Pre-computed Masks
|
24 |
|
25 |
The following masks were trained and provided by [@VainF](https://github.com/VainF). We use ``huggingface_hub`` to automatically download those masks and apply them to offcical LLMs for evaluation. Those mask files were compressed using [numpy.savez_compressed](tool_compress_mask.py). More results for baselines (SparseGPT, Wanda) can be found in the appendix.
|
26 |
-
| Model | Pattern | Training Data | Training/Eval SeqLen | PPL (Dense) | PPL (
|
27 |
-
| :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
28 |
-
| LLaMA-2 7B | 2:4 | C4 (2B Tokens)| 4096 | 5.12 | 6.78 | [HuggingFace](https://huggingface.co/Vinnnf/LLaMA-2-7B-MaskLLM-C4) |
|
29 |
-
| LLaMA-3 8B | 2:4 | C4 (2B Tokens) | 4096 | 5.75 | 8.49 | [HuggingFace]() |
|
30 |
-
| LLaMA-3.1 8B | 2:4 | C4 (2B Tokens) | 4096 | - | Comming Soon |
|
31 |
-
|
32 |
|
33 |
## How to use it
|
34 |
|
|
|
23 |
## Pre-computed Masks
|
24 |
|
25 |
The following masks were trained and provided by [@VainF](https://github.com/VainF). We use ``huggingface_hub`` to automatically download those masks and apply them to offcical LLMs for evaluation. Those mask files were compressed using [numpy.savez_compressed](tool_compress_mask.py). More results for baselines (SparseGPT, Wanda) can be found in the appendix.
|
26 |
+
| Model | Pattern | Training Data | Training/Eval SeqLen | PPL (Dense) | PPL (SparseGPT) | **PPL (MaskLLM)** | Link |
|
27 |
+
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
28 |
+
| LLaMA-2 7B | 2:4 | C4 (2B Tokens)| 4096 | 5.12 | 10.42 | **6.78** | [HuggingFace](https://huggingface.co/Vinnnf/LLaMA-2-7B-MaskLLM-C4) |
|
29 |
+
| LLaMA-3 8B | 2:4 | C4 (2B Tokens) | 4096 | 5.75 | 17.64 | **8.49** | [HuggingFace]() |
|
30 |
+
| LLaMA-3.1 8B | 2:4 | C4 (2B Tokens) | 4096 | - | - | - | Comming Soon |
|
|
|
31 |
|
32 |
## How to use it
|
33 |
|