Qwen2.5-Gutenberg-Doppel-14B
Qwen/Qwen2.5-14B-Instruct finetuned on jondurbin/gutenberg-dpo-v0.1 and nbeerbower/gutenberg2-dpo.
Method
ORPO tuned with 4x A40 for 3 epochs.
Thank you @ParasiticRogue for sponsoring.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 32.30 |
| IFEval (0-Shot) | 80.91 |
| BBH (3-Shot) | 48.24 |
| MATH Lvl 5 (4-Shot) | 0.00 |
| GPQA (0-shot) | 11.07 |
| MuSR (0-shot) | 10.02 |
| MMLU-PRO (5-shot) | 43.57 |
- Downloads last month
- 52
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
Base model
Qwen/Qwen2.5-14B
Finetuned
Qwen/Qwen2.5-14B-Instruct
Datasets used to train nbeerbower/Qwen2.5-Gutenberg-Doppel-14B
Evaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard80.910
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard48.240
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard0.000
- acc_norm on GPQA (0-shot)Open LLM Leaderboard11.070
- acc_norm on MuSR (0-shot)Open LLM Leaderboard10.020
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard43.570