---
library_name: transformers
tags:
- qwen2.5
- simpo
- sft
- unsloth
- llama-factory
- awq
- philosophical
- esoteric
- multilingual
base_model:
- Qwen/Qwen2.5-32B-Instruct
language:
- zho
- eng
- fra
- spa
- por
- deu
- ita
- rus
- jpn
- kor
- vie
- tha
- ara
---

# Model Card for forerunner-qwen32b-simpo-awq

<img src="https://huggingface.co/radm/forerunner-qwen32b-simpo-awq/resolve/main/model-image.png" alt="Model Image" width="100%">

## Model Details

### Model Description

This model is a fine-tuned version of `Qwen/Qwen2.5-32B-Instruct`. Training was conducted in two stages:
1.  **Supervised Fine-Tuning (SFT)** using the `Unsloth` library on a custom dataset.
2.  **Reinforcement Learning (RL)** using the **SimPO** algorithm (via `LLaMA-Factory`) on a preference dataset.

The model focuses on **philosophical and esoteric** topics and is multilingual. The final version is quantized using **AWQ**.

- **Developed by:** radm
- **Finetuned from model:** `Qwen/Qwen2.5-32B-Instruct`
- **Model type:** Causal LM based on the Qwen2.5 architecture
- **Language(s):** Multilingual
- **License:** Apache 2.0 (inherited from base model)

## Uses

**Example Output (with temperature=0.6, repetition_penalty=1.15):**

*User:* As a result of what mental process does duality appear in a person?

*Model:* Duality, or the concept of two-polarity, is born from the mind's ability to categorize and contrast things. The mind exercises this by distinguishing between 'mine' and 'not mine,' which leads to hierarchical thinking and the creation of concepts like 'good and evil,' 'truth and falsehood.' This dualistic nature of thought becomes ingrained over time as we encounter different experiences that further reinforce these distinctions.


### Out-of-Scope Use

The model is not designed for generating harmful, unethical, biased, or factually incorrect content. Performance on tasks outside its training domain (philosophical/esoteric chat) may be suboptimal.

## Bias, Risks, and Limitations

The model inherits biases from its base model (`Qwen/Qwen2.5-32B-Instruct`) and the fine-tuning datasets. It may generate plausible-sounding but incorrect or nonsensical information, especially on complex topics. Its "understanding" is based on patterns in the data, not genuine comprehension or consciousness. Use the outputs with critical judgment.

## Training Details

### Training Data

The model was fine-tuned in two stages:

1.  **SFT:** Used the custom dataset.
2.  **SimPO RL:** Used the preference datasets, containing pairs of preferred and rejected responses for given prompts, focusing on philosophical and esoteric themes.

### Training Procedure

#### Stage 1: Supervised Fine-Tuning (SFT)

Training was performed using the `Unsloth` library integrated with `trl`'s `SFTTrainer`.

- **Framework:** Unsloth + SFTTrainer
- **Base Model:** `Qwen/Qwen2.5-32B-Instruct`
- **LoRA Configuration:**
    - `r`: 512
    - `lora_alpha`: 512
    - `lora_dropout`: 0.0
    - `bias`: "none"
    - `target_modules`: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
    - `use_rslora`: True
- **Precision:** Auto (bfloat16 / float16)
- **Quantization (load):** 4-bit
- **Optimizer:** Paged AdamW 8-bit
- **Learning Rate:** 8e-5
- **LR Scheduler:** Cosine
- **Warmup Steps:** 10
- **Batch Size (per device):** 1
- **Gradient Accumulation Steps:** 128 (Effective Batch Size: 128)
- **Max Sequence Length:** 8192
- **Epochs:** 1

#### Stage 2: Reinforcement Learning (SimPO)

RL fine-tuning was performed using `LLaMA-Factory` and the SimPO algorithm.

- **Framework:** LLaMA-Factory + SimPO
- **Base Model:** Result of SFT stage (`Qwen/Qwen2.5-32B-Instruct-sft`)
- **LoRA Configuration:**
    - `r`: 256
    - `lora_alpha`: 256
    - `lora_dropout`: 0.0
    - `lora_target`: all
    - `use_dora`: True
    - `use_rslora`: True
- **Precision:** bfloat16
- **Quantization (load):** 4-bit
- **Optimizer:** AdamW (with `weight_decay: 0.01`)
- **Learning Rate:** 7e-7
- **LR Scheduler:** Cosine
- **Warmup Steps:** 16
- **Batch Size (per device):** 1
- **Gradient Accumulation Steps:** 64 (Effective Batch Size: 64)
- **Max Sequence Length:** 6600
- **Epochs:** 1.0

#### Stage 3: AWQ Quantization

After training completion, the model was quantized using the AWQ method to optimize performance and reduce size.