--- license: cc-by-nc-4.0 language: - ro base_model: OpenLLM-Ro/RoGemma2-9b-Instruct-DPO-2025-04-23 datasets: - OpenLLM-Ro/ro_dpo_helpsteer - OpenLLM-Ro/ro_dpo_ultrafeedback - OpenLLM-Ro/ro_dpo_magpie - OpenLLM-Ro/ro_dpo_argilla_magpie - OpenLLM-Ro/ro_dpo_helpsteer2 tags: - llama-cpp - gguf-my-repo model-index: - name: OpenLLM-Ro/RoGemma2-9b-Instruct-DPO-2025-04-23 results: - task: type: text-generation dataset: name: RoMT-Bench type: RoMT-Bench metrics: - type: Score value: 7.26 name: Score - type: Score value: 7.65 name: First turn - type: Score value: 6.86 name: Second turn - task: type: text-generation dataset: name: RoCulturaBench type: RoCulturaBench metrics: - type: Score value: 5.36 name: Score - task: type: text-generation dataset: name: Romanian_Academic_Benchmarks type: Romanian_Academic_Benchmarks metrics: - type: accuracy value: 59.79 name: Average accuracy - task: type: text-generation dataset: name: OpenLLM-Ro/ro_arc_challenge type: OpenLLM-Ro/ro_arc_challenge metrics: - type: accuracy value: 55.66 name: Average accuracy - type: accuracy value: 52.44 name: 0-shot - type: accuracy value: 55.7 name: 1-shot - type: accuracy value: 56.47 name: 3-shot - type: accuracy value: 55.7 name: 5-shot - type: accuracy value: 57.16 name: 10-shot - type: accuracy value: 56.47 name: 25-shot - task: type: text-generation dataset: name: OpenLLM-Ro/ro_mmlu type: OpenLLM-Ro/ro_mmlu metrics: - type: accuracy value: 64.0 name: Average accuracy - type: accuracy value: 65.2 name: 0-shot - type: accuracy value: 63.27 name: 1-shot - type: accuracy value: 63.83 name: 3-shot - type: accuracy value: 63.69 name: 5-shot - task: type: text-generation dataset: name: OpenLLM-Ro/ro_winogrande type: OpenLLM-Ro/ro_winogrande metrics: - type: accuracy value: 73.16 name: Average accuracy - type: accuracy value: 74.11 name: 0-shot - type: accuracy value: 72.53 name: 1-shot - type: accuracy value: 72.93 name: 3-shot - type: accuracy value: 73.09 name: 5-shot - task: type: text-generation dataset: name: OpenLLM-Ro/ro_hellaswag type: OpenLLM-Ro/ro_hellaswag metrics: - type: accuracy value: 64.26 name: Average accuracy - type: accuracy value: 65.9 name: 0-shot - type: accuracy value: 66.06 name: 1-shot - type: accuracy value: 62.36 name: 3-shot - type: accuracy value: 61.87 name: 5-shot - type: accuracy value: 65.11 name: 10-shot - task: type: text-generation dataset: name: OpenLLM-Ro/ro_gsm8k type: OpenLLM-Ro/ro_gsm8k metrics: - type: accuracy value: 37.8 name: Average accuracy - type: accuracy value: 16.83 name: 1-shot - type: accuracy value: 43.21 name: 3-shot - type: accuracy value: 53.37 name: 5-shot - task: type: text-generation dataset: name: OpenLLM-Ro/ro_truthfulqa type: OpenLLM-Ro/ro_truthfulqa metrics: - type: accuracy value: 63.86 name: Average accuracy - task: type: text-generation dataset: name: LaRoSeDa_binary type: LaRoSeDa_binary metrics: - type: macro-f1 value: 82.84 name: Average macro-f1 - type: macro-f1 value: 39.18 name: 0-shot - type: macro-f1 value: 96.59 name: 1-shot - type: macro-f1 value: 97.63 name: 3-shot - type: macro-f1 value: 97.97 name: 5-shot - task: type: text-generation dataset: name: LaRoSeDa_multiclass type: LaRoSeDa_multiclass metrics: - type: macro-f1 value: 65.95 name: Average macro-f1 - type: macro-f1 value: 58.94 name: 0-shot - type: macro-f1 value: 64.99 name: 1-shot - type: macro-f1 value: 68.86 name: 3-shot - type: macro-f1 value: 71.03 name: 5-shot - task: type: text-generation dataset: name: WMT_EN-RO type: WMT_EN-RO metrics: - type: bleu value: 28.16 name: Average bleu - type: bleu value: 26.89 name: 0-shot - type: bleu value: 31.18 name: 1-shot - type: bleu value: 30.65 name: 3-shot - type: bleu value: 23.91 name: 5-shot - task: type: text-generation dataset: name: WMT_RO-EN type: WMT_RO-EN metrics: - type: bleu value: 19.34 name: Average bleu - type: bleu value: 2.98 name: 0-shot - type: bleu value: 20.3 name: 1-shot - type: bleu value: 30.08 name: 3-shot - type: bleu value: 24.01 name: 5-shot - task: type: text-generation dataset: name: XQuAD type: XQuAD metrics: - type: exact_match value: 30.82 name: Average exact_match - type: f1 value: 48.53 name: Average f1 - task: type: text-generation dataset: name: STS type: STS metrics: - type: spearman value: 73.24 name: Average spearman - type: pearson value: 73.13 name: Average pearson - task: type: text-generation dataset: name: XQuAD_EM type: XQuAD_EM metrics: - type: exact_match value: 26.39 name: 0-shot - type: exact_match value: 23.87 name: 1-shot - type: exact_match value: 34.03 name: 3-shot - type: exact_match value: 38.99 name: 5-shot - task: type: text-generation dataset: name: XQuAD_F1 type: XQuAD_F1 metrics: - type: f1 value: 43.28 name: 0-shot - type: f1 value: 37.38 name: 1-shot - type: f1 value: 54.08 name: 3-shot - type: f1 value: 59.38 name: 5-shot - task: type: text-generation dataset: name: STS_Spearman type: STS_Spearman metrics: - type: spearman value: 73.46 name: 1-shot - type: spearman value: 73.55 name: 3-shot - type: spearman value: 72.7 name: 5-shot - task: type: text-generation dataset: name: STS_Pearson type: STS_Pearson metrics: - type: pearson value: 74.87 name: 1-shot - type: pearson value: 72.96 name: 3-shot - type: pearson value: 71.55 name: 5-shot --- # LuuNgoc2k2/RoGemma2-9b-Instruct-DPO-2025-04-23-Q8_0-GGUF This model was converted to GGUF format from [`OpenLLM-Ro/RoGemma2-9b-Instruct-DPO-2025-04-23`](https://huggingface.co/OpenLLM-Ro/RoGemma2-9b-Instruct-DPO-2025-04-23) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space. Refer to the [original model card](https://huggingface.co/OpenLLM-Ro/RoGemma2-9b-Instruct-DPO-2025-04-23) for more details on the model. ## Use with llama.cpp Install llama.cpp through brew (works on Mac and Linux) ```bash brew install llama.cpp ``` Invoke the llama.cpp server or the CLI. ### CLI: ```bash llama-cli --hf-repo LuuNgoc2k2/RoGemma2-9b-Instruct-DPO-2025-04-23-Q8_0-GGUF --hf-file rogemma2-9b-instruct-dpo-2025-04-23-q8_0.gguf -p "The meaning to life and the universe is" ``` ### Server: ```bash llama-server --hf-repo LuuNgoc2k2/RoGemma2-9b-Instruct-DPO-2025-04-23-Q8_0-GGUF --hf-file rogemma2-9b-instruct-dpo-2025-04-23-q8_0.gguf -c 2048 ``` Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well. Step 1: Clone llama.cpp from GitHub. ``` git clone https://github.com/ggerganov/llama.cpp ``` Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux). ``` cd llama.cpp && LLAMA_CURL=1 make ``` Step 3: Run inference through the main binary. ``` ./llama-cli --hf-repo LuuNgoc2k2/RoGemma2-9b-Instruct-DPO-2025-04-23-Q8_0-GGUF --hf-file rogemma2-9b-instruct-dpo-2025-04-23-q8_0.gguf -p "The meaning to life and the universe is" ``` or ``` ./llama-server --hf-repo LuuNgoc2k2/RoGemma2-9b-Instruct-DPO-2025-04-23-Q8_0-GGUF --hf-file rogemma2-9b-instruct-dpo-2025-04-23-q8_0.gguf -c 2048 ```