Qwen 2.5 Fine-Tuned Model (pirate-qwen2.5-finetuned)

Model Overview

This is a fine-tuned version of the Qwen 2.5 LLM, optimized for text generation tasks with a focus on conversational abilities. Fine-tuning was performed using Supervised Fine-Tuning (SFT) on a custom dataset.

• Base Model: Qwen 2.5
• Fine-Tuning Method: SFT
• Hardware: A100 GPU with BF16 precision
• Training Epochs: 3
• Learning Rate: 2.0e-04
• Batch Size: 4 (with gradient accumulation of 4)

Training Data

The model was fine-tuned on a custom dataset (not publicly available) containing conversational text. The dataset was preprocessed to ensure quality and relevance.

Performance

• Evaluation Metrics: [Add your metrics, e.g., perplexity, BLEU score, or accuracy]
• Qualitative Results: Improved coherence and relevance in conversational tasks compared to the base Qwen 2.5 model.

Usage

Load the Model

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("sahibsingh12/pirate-qwen2.5-finetuned")
tokenizer = AutoTokenizer.from_pretrained("sahibsingh12/pirate-qwen2.5-finetuned")

# Generate text
prompt = "Tell me about AI."
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))