chhatramani/Gemma3-4B-Nepal-Civilcode-v1

Model Card

This model is a finetuned version of Google's Gemma 3.4B, specifically optimized for Question Answering (QA) tasks related to Nepal Civil Law. It has been trained on a unique bilingual dataset, enabling it to answer questions in both Nepali and English.

Model Description

chhatramani/Gemma3-4B-Nepal-Civilcode-v1 is a specialized Large Language Model (LLM) designed to provide accurate and relevant answers concerning the Civil Code of Nepal. By leveraging the powerful Gemma 3.4B architecture and finetuning it on a curated dataset of civil law QA pairs, this model aims to bridge the gap in legal AI resources for the Nepali context. Its bilingual capability makes it a versatile tool for a wider audience, including legal professionals, researchers, students, and the general public.

Intended Use

This model is primarily intended for:

• Question Answering: Answering specific questions about the Nepal Civil Code in both Nepali and English.
• Legal Information Retrieval: Assisting in quickly finding information within the vast domain of Nepali civil law.
• Educational Purposes: Serving as a tool for learning and understanding the provisions of the Nepal Civil Code.
• AI-powered Legal Assistants: As a component in building more comprehensive legal AI applications.

Training Data

The model was finetuned on a comprehensive bilingual dataset comprising QA pairs derived from Nepal's Civil Law. The datasets used are:

Nepali Civil Law QA Dataset: chhatramani/nepali_muluki_dewani_QA_v1

• Size: 3.52K unique Question-Answering pairs.
• Content: Questions and answers specifically in Nepali, extracted from and related to the Muluki Dewani Samhita (Civil Code) of Nepal.

English Civil Law QA Dataset: chhatramani/nepal_civil_law_QA_v2

• Size: 2.71K unique Question-Answering pairs.
• Content: Questions and answers in English, covering various aspects of Nepal's Civil Law.

The combination of these datasets ensures a robust understanding of the subject matter across both languages.

Training Details

• Base Model: Google Gemma 3.4B
• Finetuning Objective: Question Answering
• Average Training Loss: 0.6
• Framework: Hugging Face Transformers
• Training Hardware: (Optional: Add details if known, e.g., "NVIDIA A100 GPUs")
• Training Software: (Optional: Add details if known, e.g., "PyTorch, accelerate")

How to Use

You can easily load and use this model with the Hugging Face transformers library.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the tokenizer and model
model_name = "chhatramani/Gemma3-4B-Nepal-Civilcode-v1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)

# Move model to GPU if available
if torch.cuda.is_available():
    model = model.to("cuda")

def generate_answer(question, max_new_tokens=200):
    # For instruction-tuned models, it's good practice to wrap the prompt
    # in an instruction format. Adjust based on how your data was formatted.
    prompt = f"Question: {question}\nAnswer:"
    input_ids = tokenizer(prompt, return_tensors="pt").input_ids
    
    if torch.cuda.is_available():
        input_ids = input_ids.to("cuda")

    outputs = model.generate(
        input_ids,
        max_new_tokens=max_new_tokens,
        num_return_sequences=1,
        pad_token_id=tokenizer.eos_token_id,
        do_sample=True, # Set to True for more varied responses
        top_k=50,
        top_p=0.95,
        temperature=0.7
    )
    # Decode only the newly generated tokens
    decoded_output = tokenizer.decode(outputs[0][len(input_ids[0]):], skip_special_tokens=True)
    return decoded_output.strip()

# Example Usage in Nepali
nepali_question = "नेपालको संविधानको धारा ३१ मा के व्यवस्था छ?"
nepali_answer = generate_answer(nepali_question)
print(f"Nepali Question: {nepali_question}")
print(f"Nepali Answer: {nepali_answer}\n")

# Example Usage in English
english_question = "What are the provisions regarding property rights in the Nepal Civil Code?"
english_answer = generate_answer(english_question)
print(f"English Question: {english_question}")
print(f"English Answer: {english_answer}")

Limitations and Bias

While chhatramani/Gemma3-4B-Nepal-Civilcode-v1 is a powerful tool, it's important to acknowledge its limitations:

• Hallucination: Like all LLMs, this model may occasionally generate incorrect, nonsensical, or hallucinated information. Always verify critical legal information with official sources.
• Scope: The model's knowledge is primarily limited to the Nepal Civil Code as represented in its training data. It may not be proficient in other legal domains (e.g., criminal law, constitutional law beyond civil aspects) or recent amendments not present in the dataset.
• Nuance and Interpretation: Legal interpretation often requires human judgment, context, and a deep understanding of specific case facts. This model should not be used as a substitute for professional legal advice.
• Bias: The model may inherit biases present in its base model or the training data. Efforts have been made to use authoritative sources, but inherent biases in language or legal phrasing might still exist.
• Performance Variation: While bilingual, the quality of responses might vary between Nepali and English depending on the complexity of the query and the density of relevant training data in each language.

Acknowledgements

• We extend our gratitude to Google for developing and open-sourcing the Gemma models.

Citation

If you use this model in your research or application, please consider citing the original Gemma model and the datasets used for finetuning.

@article{gemma2024,
  title={Gemma: Open Models from Google},
  author={Google},
  year={2024},
  url={https://blog.google/technology/ai/gemma-open-models-deepmind/ }
}