GPTQ-for-SantaCoder
Visit GPTQ-for-SantaCoder for instructions on how to use the model weights here. If you want 4-bit weights, visit santacoder-GPTQ-4bit-128g.
Results
| SantaCoder | Bits | group-size | memory(MiB) | wikitext2 | ptb | c4 | stack | checkpoint size(MB) |
|---|---|---|---|---|---|---|---|---|
| FP32 | 32 | - | 4344.722 | 24.927 | 38.574 | 27.779 | 2.619 | 4394 |
| BF16 | 16 | - | 2173.680 | 24.960 | 38.597 | 27.794 | 2.621 | 2195 |
| GPTQ | 8 | -1 | 1396.548 | 24.936 | 38.592 | 27.785 | 2.619 | 1411 |
| GPTQ | 4 | -1 | 911.384 | 26.581 | 40.717 | 29.232 | 2.658 | 913 |
| GPTQ | 3 | -1 | - | 11761.473 | 7273.338 | 9124.941 | 2485.844 | 789 |
| GPTQ | 2 | -1 | - | 67976.797 | 68994.484 | 73294.438 | 45370.488 | 649 |
License
The model is licenses under the CodeML Open RAIL-M v0.1 license. You can find the full license here.
Acknowledgements
Thanks to everyone in BigCode who worked so hard to create these code models.
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
HF Inference deployability: The model has no library tag.