|
|
--- |
|
|
license: gemma |
|
|
base_model: |
|
|
- google/gemma-3-1b-it |
|
|
base_model_relation: quantized |
|
|
pipeline_tag: text-generation |
|
|
--- |
|
|
This version is quantized by h4shy with the consideration of low-end hardware and CPU-only setups. The goal here is to achieve an inference-ready setup aiming for production use with considerable resource constrains, as these particular quantization choices will help with running inferences where there are medium to high CPU constrains and low to medium RAM constrains, as well as reserving resources for production efficiency. |
|
|
|
|
|
Q5_0: Medium to fast inference, optimal RAM usage. |
|
|
Q8_0: More inference speed, more RAM usage. |
|
|
|
|
|
Evaluations and precise research coming soon. |
|
|
|
|
|
Original model: [gemma-3-1b-it](https://huggingface.co/google/gemma-3-1b-it) |
|
|
Software used for quantization: [llama.cpp](https://github.com/ggml-org/llama.cpp) |
