sumink/enhigh

Model: LLaMA (IFD Top 30% + High Instruction Entropy)

🔍 Purpose

Fine-tune meta-llama/Llama-3.2-1B on instructions with high entropy among the high-IFD group.
These instructions are more diverse and complex in word usage.

📂 Dataset

• alpaca2000_entropy_high.csv
  • From alpaca2000.csv (IFD 상위 30%)
  • instruction entropy 상위 30% 추출 (약 180개)

⚙️ Training Config

• Model: meta-llama/Llama-3.2-1B
• Precision: bf16 or float32
• Epochs: 3
• Max length: 2048
• Output: output/llama_entropy_high

🧪 Goal

Evaluate whether complex and diverse instructions improve the fine-tuning performance on instruction-following tasks.