mohammed-orabi2/qwen-poetry-arabic-lora

Model Card for Model ID

Model ID: mohammed-orabi2/qwen-poetry-lora2

Model Details

Model Description: This is a LoRA fine-tuned version of the Qwen/Qwen3-1.7B model, specifically trained to generate Arabic poetic responses in a conversational format. It was trained on a dataset of 1,000 synthetic Arabic poetry dialogues, each containing a user query and a poetic response.

Developed by: Mohammed Orabi

Shared by : mohammed-orabi2

Model type: Causal Language Model with LoRA adaptation

Language(s) (NLP): Arabic

License: Apache 2.0 (inherits from Qwen3-1.7B)

Finetuned from model : Qwen/Qwen3-1.7B

**Model Sources ** Repository: https://huggingface.co/mohammed-orabi2/qwen-poetry-lora2

Uses

Direct Use: This model can be used for generating Arabic poetry in response to user queries, particularly in cultural, educational, or creative chatbot applications.

Downstream Use :

Poetry recommendation systems
Arabic literature generation tools
Creative writing assistants

Out-of-Scope Use:

Non-Arabic generation tasks
Factual or knowledge-based QA tasks
Sensitive or safety-critical environments

Bias, Risks, and Limitations

The model was fine-tuned on synthetic poetic data and may:

Favor specific poetic structures
Fail on factual, political, or philosophical prompts
Generate romantic or metaphorical content that could be misinterpreted in serious contexts

Users should avoid relying on this model for objective or critical outputs.

Recommendations

Users (both direct and downstream) should be aware of the creative, poetic intent of this model. For factual content, use general-purpose LLMs. Evaluate outputs manually before publishing or broadcasting.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B", device_map="auto", torch_dtype=torch.float16)
model = PeftModel.from_pretrained(base_model, "mohammed-orabi2/qwen-poetry-arabic-lora")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")

prompt = "اكتب لي بيت شعر عن النجاح."
chat = [{"role": "user", "content": prompt}]
formatted_prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(formatted_prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

Training Data: 1,000 synthetic Arabic poetic dialogues (prompt + poetic response) generated programmatically.

Preprocessing :

Applied Qwen chat template
Tokenized using Qwen3-1.7B tokenizer with padding/truncation

Training Hyperparameters:

Epochs: 5
Batch size: 2
Max length: 1024
Learning rate: 2e-4
LoRA config: r=8, alpha=16, dropout=0.05, target: ["q_proj", "v_proj"]

Speeds, Sizes, Times :

Training time: ~24 minutes on L4 GPU
Model size: LoRA adapter ~100MB

Evaluation

Testing Data: 50 reserved samples from the poetic dataset

Factors:

Response fluency
Arabic poetic structure
Topical relevance

Metrics:

Manual review (subjective)
BLEU/Rouge not applicable

Results:

90% generated responses respected rhyme/meter and matched prompt topics

Summary

Model Examination [optional]: Output behavior consistent with training intent. Performs well within poetic use-case boundaries.

Environmental Impact

Hardware Type: NVIDIA L4 Hours used: ~0.4 hours (24 minutes) Cloud Provider: Google Colab Compute Region: US (GCP default) Carbon Emitted: Estimated ~0.2 kg CO2e

Technical Specifications

Model Architecture and Objective: Transformer decoder (CausalLM) + LoRA injection

Compute Infrastructure: Google Colab

Hardware: NVIDIA L4 (24 mins)

Software:

Transformers 4.x
PEFT 0.15.2
Accelerate 0.25+

Citation

BibTeX:

@misc{qwenpoetry2025,
  author = {Mohammed Orabi},
  title = {Qwen Arabic Poetry LoRA},
  year = {2025},
  howpublished = {\url{https://huggingface.co/mohammed-orabi2/qwen-poetry-lora2}}
}

APA: Mohammed Orabi. (2025). Qwen Arabic Poetry LoRA [Model]. Hugging Face. https://huggingface.co/mohammed-orabi2/qwen-poetry-lora2

Glossary

LoRA: Low-Rank Adaptation, a method for efficient model fine-tuning
CausalLM: Causal Language Modeling, predicts the next token in a sequence

More Information

For support or feedback, please open an issue on the Hugging Face repo or contact via Hugging Face profile.

Model Card Authors

Mohammed Orabi

Model Card Contact

https://huggingface.co/mohammed-orabi2

Framework versions

Transformers: 4.x
PEFT: 0.15.2
Datasets: latest
Accelerate: 0.25+

mohammed-orabi2
/

qwen-poetry-arabic-lora