english-hindi-colloquial-translator

This model is a fine-tuned version of unsloth/tinyllama-chat-bnb-4bit on the English-Hindi Colloquial Dataset (Sadiya025/english-hindi-colloquial-dataset). It achieves the following results on the evaluation set:

• Loss: 12.2561

Model description

Base Model: unsloth/tinyllama-chat-bnb-4bit Dataset: Sadiya025/english-hindi-colloquial-dataset Task: English-to-Hindi Colloquial Translation Quantization: 4-bit for optimized inference

How to Use

To generate hindi colloquial translations, run inference.py from this repository. The script automatically handles model loading, tokenization, and inference.

git clone https://huggingface.co/Sadiya025/english-hindi-colloquial-translator
cd english-hindi-colloquial-translator
python inference.py

Training and evaluation data

The model was fine-tuned on the Sadiya025/english-hindi-colloquial-dataset, which consists of English sentences and their corresponding colloquial Hindi translations.

Training procedure

• Base Model: unsloth/tinyllama-chat-bnb-4bit
• Frameworks:
  • PEFT 0.14.0
  • Transformers 4.49.0
  • PyTorch 2.6.0+cu124
  • Datasets 3.2.0
  • Tokenizers 0.21.0
• Fine-tuning Method: Parameter-Efficient Fine-Tuning (PEFT)
• Precision: 4-bit quantized model for memory efficiency

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
2.1854	0.2	500	6.7063
2.2112	0.4	1000	6.5494
2.1413	0.6	1500	6.5787
2.1303	0.8	2000	6.6641
1.9676	1.0	2500	6.6817
1.9816	1.2	3000	6.7549
2.2404	1.4	3500	6.7187
2.0038	1.6	4000	6.7696
1.9079	1.8	4500	6.9118
2.1682	2.0	5000	6.9245
1.8931	2.2	5500	7.3770
1.9293	2.4	6000	8.2341
2.0817	2.6	6500	10.7503
2.1382	2.8	7000	12.0544
1.8867	3.0	7500	12.2561

Framework versions

• PEFT 0.14.0
• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0