# Model: LLaMA (IFD Top 30% + Low Instruction Entropy)

## 🔍 Purpose

Fine-tune `meta-llama/Llama-3.2-1B` on instructions with **low entropy** among the high-IFD group.  
These instructions tend to be **short, simple, and repetitive**.

## 📂 Dataset

- `alpaca2000_entropy_low.csv`
  - From `alpaca2000.csv` (IFD 상위 30%)
  - instruction entropy 하위 30% 추출 (약 180개)

## ⚙️ Training Config

- Model: `meta-llama/Llama-3.2-1B`
- Precision: `bf16` or `float32`
- Epochs: 3
- Max length: 2048
- Output: `output/llama_entropy_low`

## 🧪 Goal

Measure whether **simpler instructions** lead to better model learning by reducing prompt complexity.