Redgerd/llama3-roman-urdu-finetuned
Redgerd/llama3-roman-urdu-finetuned is a multilingual instruction-tuned LLaMA 3 model fine-tuned on a combination of Roman Urdu QA pairs and English examples from the Stanford Alpaca dataset.
This model is designed to enhance performance in low-resource, multilingual, and instruction-following tasks, especially involving Roman Urdu.
Model Details
- Base Model: Meta LLaMA 3
- Architecture: Decoder-only transformer
- Fine-tuned On: Custom Alpaca-style dataset
- Languages: Roman Urdu π΅π° & English π¬π§
- Format: Instruction-tuning (compatible with Alpaca style)
Dataset Overview
A custom dataset with ~500 instruction-based examples in Roman Urdu and ~500 from Stanford Alpaca at from Redgerd/roman-urdu-alpaca-qa-mix
Training Setup
- Frameworks:
transformers
,unsloth
- Format: Instruction-based fine-tuning (
instruction
,input
,output
) - Environment: A100 GPU with bfloat16 precision
- Checkpointing: Supported
Credits
Developed by Muhammad Salaar using:
- Stanford Alpaca Dataset
- GPT-4 for Roman Urdu generation
- LLaMA 3.1 as the base model
For feedback or collaboration, visit: github.com/Redgerd
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
π
Ask for provider support