gemma-2-2b-lean-expert-optimized

Optimized Gemma Model for 94%+ Success Rate

This repository contains the training configuration for an optimized Gemma-2-2B model targeting 94%+ success rate on Lean trading algorithm optimization tasks.

Training Configuration

• Base Model: google/gemma-2-2b
• Dataset: Kronu/lean-expert-optimized-2000
• Target Success Rate: 94%+
• Expected Performance: 96% (94-98% range)

Key Optimizations

• JSON Parsing Focus: 1,333 examples (0% → 95% success target)
• Enhanced LoRA: rank=64, alpha=128
• Optimized Training: 12 epochs, 2e-4 learning rate
• Advanced Configuration: Gradient checkpointing, FP16

Training Instructions

To train this model using HuggingFace Jobs:

1. Set up your HuggingFace token as environment variable
2. Run the training script: python train.py
3. Monitor training progress in the HuggingFace dashboard

Expected Results

• Training Time: 25-35 minutes
• Cost: $3-5
• Final Model: Kronu/gemma-2-2b-lean-expert-optimized
• Success Rate: 96% (94-98% range)

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load the trained model
base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2-2b")
model = PeftModel.from_pretrained(base_model, "Kronu/gemma-2-2b-lean-expert-optimized")
tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b")