Vinnnf commited on
Commit
b8e7608
·
verified ·
1 Parent(s): 1e4ced4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -6
README.md CHANGED
@@ -23,12 +23,11 @@ pip install transformers accelerate datasets SentencePiece
23
  ## Pre-computed Masks
24
 
25
  The following masks were trained and provided by [@VainF](https://github.com/VainF). We use ``huggingface_hub`` to automatically download those masks and apply them to offcical LLMs for evaluation. Those mask files were compressed using [numpy.savez_compressed](tool_compress_mask.py). More results for baselines (SparseGPT, Wanda) can be found in the appendix.
26
- | Model | Pattern | Training Data | Training/Eval SeqLen | PPL (Dense) | PPL (Sparse) | Link |
27
- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
28
- | LLaMA-2 7B | 2:4 | C4 (2B Tokens)| 4096 | 5.12 | 6.78 | [HuggingFace](https://huggingface.co/Vinnnf/LLaMA-2-7B-MaskLLM-C4) |
29
- | LLaMA-3 8B | 2:4 | C4 (2B Tokens) | 4096 | 5.75 | 8.49 | [HuggingFace]() |
30
- | LLaMA-3.1 8B | 2:4 | C4 (2B Tokens) | 4096 | - | Comming Soon |
31
-
32
 
33
  ## How to use it
34
 
 
23
  ## Pre-computed Masks
24
 
25
  The following masks were trained and provided by [@VainF](https://github.com/VainF). We use ``huggingface_hub`` to automatically download those masks and apply them to offcical LLMs for evaluation. Those mask files were compressed using [numpy.savez_compressed](tool_compress_mask.py). More results for baselines (SparseGPT, Wanda) can be found in the appendix.
26
+ | Model | Pattern | Training Data | Training/Eval SeqLen | PPL (Dense) | PPL (SparseGPT) | **PPL (MaskLLM)** | Link |
27
+ | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
28
+ | LLaMA-2 7B | 2:4 | C4 (2B Tokens)| 4096 | 5.12 | 10.42 | **6.78** | [HuggingFace](https://huggingface.co/Vinnnf/LLaMA-2-7B-MaskLLM-C4) |
29
+ | LLaMA-3 8B | 2:4 | C4 (2B Tokens) | 4096 | 5.75 | 17.64 | **8.49** | [HuggingFace]() |
30
+ | LLaMA-3.1 8B | 2:4 | C4 (2B Tokens) | 4096 | - | - | - | Comming Soon |
 
31
 
32
  ## How to use it
33