AliMuhammad73/testing-model

Custom Urdu LLM

This is a custom transformer-based Large Language Model for Urdu.

Model Details

• Architecture: Transformer (GPT-based)
• Framework: PyTorch
• Tokenizer: SentencePiece
• Hyperparameters:
  • Vocabulary Size: 20,000
  • Embedding Size: 768
  • Attention Heads: 12
  • Layers: 12
  • Dropout: 0.2

Usage

First you will need to download the modeling_gpt.py file from the repo. Once that's been done, you can define another file and use the following code to generate text from the model:

from modeling_gpt import GPTLanguageModel
from transformers import AutoTokenizer

model = GPTLanguageModel.from_pretrained("AliMuhammad73/testing-model")
tokenizer = AutoTokenizer.from_pretrained("AliMuhammad73/testing-model")

# sentence in urdu
prompt = "پاکستان ایک ایسا ملک ہے جو جنوبی ایشیا میں واقع ہے۔ اس کی سرحدیں ہندوستان، چین، افغانستان، اور "
encoded = tokenizer.encode(prompt)
encoded_tensor = torch.tensor(encoded).unsqueeze(0)
output = model.generate(encoded_tensor, max_new_tokens=64)

response = tokenizer.decode(output[0].squeeze().tolist())

---

license: apache-2.0