Quantize
- https://www.kaggle.com/code/reginliu/causallm2gguf-imatrix
- https://colab.research.google.com/drive/1JT3XFjD7CTRB97pu3QpeGuzWA1yYEAM7?usp=sharing
Perplexity
- causallm_14b.IQ4_XS.gguf: PPL = 13.4127 +/- 0.13762
- causallm_14b.IQ3_XS.gguf: PPL = 13.3798 +/- 0.13641
- causallm_14b.IQ2_XXS.gguf: PPL = 15.0160 +/- 0.15004
- Downloads last month
- 30
Hardware compatibility
Log In
to view the estimation
2-bit
3-bit
4-bit
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support