---
library_name: peft
license: apache-2.0
base_model: unsloth/tinyllama-chat-bnb-4bit
tags:
- unsloth
- generated_from_trainer
model-index:
- name: english-telugu-colloquial-translator
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# english-telugu-colloquial-translator

This model is a fine-tuned version of [unsloth/tinyllama-chat-bnb-4bit](https://huggingface.co/unsloth/tinyllama-chat-bnb-4bit) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 9.2516

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2
- num_epochs: 10
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 14.6072       | 0.2   | 2    | 9.2992          |
| 14.7062       | 0.4   | 4    | 9.2992          |
| 14.7707       | 0.6   | 6    | 9.2992          |
| 14.7094       | 0.8   | 8    | 9.2992          |
| 14.722        | 1.0   | 10   | 9.2992          |
| 14.6891       | 1.2   | 12   | 9.2992          |
| 14.7003       | 1.4   | 14   | 9.2992          |
| 14.7556       | 1.6   | 16   | 9.2994          |
| 10.6826       | 1.8   | 18   | 9.2998          |
| 9.8697        | 2.0   | 20   | 9.3005          |
| 11.1377       | 2.2   | 22   | 9.3005          |
| 7.7884        | 2.4   | 24   | 9.3002          |
| 6.0632        | 2.6   | 26   | 9.2997          |
| 5.2584        | 2.8   | 28   | 9.2984          |
| 4.8989        | 3.0   | 30   | 9.2961          |
| 4.6178        | 3.2   | 32   | 9.2924          |
| 4.4081        | 3.4   | 34   | 9.2877          |
| 4.1896        | 3.6   | 36   | 9.2824          |
| 4.1095        | 3.8   | 38   | 9.2774          |
| 3.998         | 4.0   | 40   | 9.2726          |
| 3.9535        | 4.2   | 42   | 9.2671          |
| 3.9242        | 4.4   | 44   | 9.2624          |
| 3.9082        | 4.6   | 46   | 9.2579          |
| 3.8371        | 4.8   | 48   | 9.2548          |
| 3.8282        | 5.0   | 50   | 9.2529          |
| 3.7675        | 5.2   | 52   | 9.2508          |
| 3.8366        | 5.4   | 54   | 9.2504          |
| 3.8349        | 5.6   | 56   | 9.2502          |
| 3.76          | 5.8   | 58   | 9.2492          |
| 3.7513        | 6.0   | 60   | 9.2491          |
| 3.7074        | 6.2   | 62   | 9.2493          |
| 3.6912        | 6.4   | 64   | 9.2487          |
| 3.7123        | 6.6   | 66   | 9.2474          |
| 3.6827        | 6.8   | 68   | 9.2481          |
| 3.68          | 7.0   | 70   | 9.2487          |
| 3.664         | 7.2   | 72   | 9.2483          |
| 3.7076        | 7.4   | 74   | 9.2482          |
| 3.6248        | 7.6   | 76   | 9.2486          |
| 3.6861        | 7.8   | 78   | 9.2492          |
| 3.6525        | 8.0   | 80   | 9.2495          |
| 3.6564        | 8.2   | 82   | 9.2499          |
| 3.6094        | 8.4   | 84   | 9.2502          |
| 3.5723        | 8.6   | 86   | 9.2505          |
| 3.5466        | 8.8   | 88   | 9.2506          |
| 3.5512        | 9.0   | 90   | 9.2508          |
| 3.5659        | 9.2   | 92   | 9.2508          |
| 3.6072        | 9.4   | 94   | 9.2510          |
| 3.5422        | 9.6   | 96   | 9.2511          |
| 3.5725        | 9.8   | 98   | 9.2514          |
| 3.5284        | 10.0  | 100  | 9.2516          |


### Framework versions

- PEFT 0.14.0
- Transformers 4.48.3
- Pytorch 2.6.0+cu124
- Datasets 3.3.1
- Tokenizers 0.21.0