Model Card for oscarstories/lorastral24b_0604

Model Description

This model is a fine-tuned version of the Mistral Small 24B Instruct model, optimized for child-friendly language generation. It produces safe, age-appropriate stories and dialogues for educational applications, especially for children aged 6 to 12.

Model Details

• Developed by: HeyQQ GmbH

• Model type: Causal Language Model (transformer-based)

• Language(s):

  • German (primary)
  • English (secondary)

• License: Apache-2.0

• Finetuned from model: mistralai/Mistral-Small-24B-Instruct-2501

Uses

Direct Use

Suitable for generating child-appropriate stories, dialogues, and learning content in educational applications, especially for German-speaking regions.

Out-of-Scope Use

Not intended for critical applications such as medical, legal, or safety-critical advice. Human moderation is recommended for sensitive content.

Bias, Risks, and Limitations

The model was trained with a focus on safe, inclusive, and non-violent content. Unintended errors are possible, especially with complex or ambiguous inputs.

Recommendations

Users should be aware of potential risks and ensure human oversight, particularly when deploying in production or sensitive environments.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained("oscarstories/lorastral24b_0604")
model = AutoModelForCausalLM.from_pretrained("oscarstories/lorastral24b_0604")

generator = pipeline("text-generation", model=model, tokenizer=tokenizer)

output = generator("Tell a short story for children about sharing.", max_new_tokens=200)
print(output[0]["generated_text"])

Training Details

Training Data

• Handcrafted data (proprietary): Curated texgts for children aged 6–12
• Public Domain and Creative Commons children’s books and texts

Preprocessing

• Strict filtering for age-appropriateness (CEFR A1–A2 level)
• Positive tone, moral orientation
• Gender-balanced and inclusive language

Training Hyperparameters

• Training regime: bf16 mixed precision

Hardware

• NVIDIA H100 GPUs (2 x 80 GB)

Other Training Info

• VRAM required: 47.14 GB
• Throughput: 7.1k tokens/second
• Training duration: 100 hours

Environmental Impact

• Hardware Type: NVIDIA H100 (2 GPUs)
• Hours used: 100

---