|
--- |
|
base_model: unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit |
|
tags: |
|
- text-generation-inference |
|
- transformers |
|
- unsloth |
|
- qwen2 |
|
- gguf |
|
license: apache-2.0 |
|
language: |
|
- en |
|
- es |
|
datasets: |
|
- Kukedlc/dpo-orpo-spanish-15k |
|
library_name: transformers |
|
--- |
|
[<img src="https://cdn-avatars.huggingface.co/v1/production/uploads/67b2f4e49edebc815a3a4739/R1g957j1aBbx8lhZbWmxw.jpeg" width="200"/>](https://huggingface.co/fjmgAI) |
|
|
|
## Fine-Tuned Model |
|
|
|
**`fjmgAI/b1-R1-Zero-3B-GGUF`** |
|
|
|
## Base Model |
|
**`unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit`** |
|
|
|
## Fine-Tuning Method |
|
Fine-tuning was performed using **[`unsloth`](https://github.com/unslothai/unsloth)**, an efficient fine-tuning framework optimized for low-resource environments and Huggingface's TRL library. |
|
|
|
## Dataset |
|
**[`Kukedlc/dpo-orpo-spanish-15k`](https://huggingface.co/datasets/Kukedlc/dpo-orpo-spanish-15k)** |
|
|
|
### Description |
|
A Spanish-language dataset containing **15,000 examples**, designed for **Direct Preference Optimization (DPO)** or **Outcome-Regularized Preference Optimization (ORPO).** |
|
|
|
### Adaptation |
|
The dataset was adapted to a reasoning-based format for GPRO, enhancing its ability to guide preference-based decision-making during fine-tuning. This adaptation ensures better alignment with instruction-following tasks in Spanish. |
|
|
|
## Fine-Tuning Details |
|
- The model was trained using the **GPRO algorithm**, leveraging structured preference data to refine its response generation. |
|
- The model was fine-tuned to maintain its **4-bit quantization (`bnb-4bit`)** for memory efficiency while aligning its outputs with the characteristics of the Spanish dataset. |
|
- The focus was on retaining the model's **instructional abilities** while improving its **understanding and generation** of Spanish text. |
|
|
|
## Purpose |
|
This fine-tuned model is intended for **Spanish-language applications** that require efficient AI that follows instructions using a **lightweight reasoning process.** |
|
|
|
|
|
- **Developed by:** fjmgAI |
|
- **License:** apache-2.0 |
|
|
|
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth) [<img src="https://camo.githubusercontent.com/9585eb3e70c8138cbc0f73de7e970be4c668e957e45d16fc3ee6687fcc1da905/68747470733a2f2f68756767696e67666163652e636f2f64617461736574732f74726c2d6c69622f646f63756d656e746174696f6e2d696d616765732f7265736f6c76652f6d61696e2f74726c5f62616e6e65725f6461726b2e706e67" width="200"/>](https://github.com/huggingface/trl?tab=readme-ov-file) |
|
|
