Zlatorog-12B-Instruct-Beta

This model is a fine-tuned version of zidsi/MistralNemoCPT6 on the custom mix of SFT datasets.

Model description

This model is based on Mistral Nemo 12B and has undergone multiple refinement stages, including Continual Pretraining (CPT) and Supervised Fine-Tuning (SFT) with an instruction-tuned objective. It is designed for high-quality instruction following, reasoning, and tool-augmented generation tasks. The model has been optimized for both language understanding and function-calling scenarios.

Intended uses & limitations

Research, explore and have fun with Slovenian LLM :)

Training and evaluation data

Bad standard Slovenian benchmarks results but sometimes impresssive "real world" prompt responses :)

Reduced hallucinations rate on "Who is ...?" prompts.

Up to 16k ctx should work OK, for longer contexts training data would be required to improve @ CPT Long stage.

🔧 Tool Usage

This model supports tool-augmented generation, allowing it to invoke external tools via structured function calls based on user prompts. The tool usage follows a special tokenized format, with tool availability declared at the beginning of the prompt and a corresponding tool call generated in the model's output.

📌 Example

Input (Prompt):

<s>[AVAILABLE_TOOLS][{"type": "function", "function": {"name": "poisci_novice", "description": "Orodje poišče današnje novice glede na iskalni niz - ključne besede v besedilu.", "parameters": {"type": "object", "properties": {"iskalni_niz": {"type": "string", "description": "Iskalni niz za iskanje novic."}}, "required": ["iskalni_niz"]}}}][/AVAILABLE_TOOLS][INST]Naredi seznam današnjih novic o velikih jezikovnih modelih.[/INST]

Model Output:

[TOOL_CALLS][{"name": "poisci_novice", "arguments": {"iskalni_niz": "veliki jezikovni modeli"}}]

🛠️ How It Works

[AVAILABLE_TOOLS]: Declares tools the model can call, formatted as JSON.
[INST]...[/INST]: Wraps the user instruction or prompt.
[TOOL_CALLS]: The model responds with structured tool calls matching the declared API.

This format is suitable for integration into multi-tool agents or custom orchestration pipelines where the model determines and delegates external functions.

📤 Tool Call Schema

Tool definitions follow the OpenAI function calling specification, using a JSON Schema to define parameters and their constraints.

License

This model is released under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.

Attribution must be given to the original author.
NonCommercial use only. Commercial use is not permitted under this license.
NoDerivatives means you may not share adaptations of the model.

For commercial use, custom development, or to request extended rights, please send an email to info <at> zid.si

More details on the license: https://creativecommons.org/licenses/by-nc-nd/4.0/

💡 Support Future Research

The developer is currently seeking funding to continue research and development on this model and related projects. If you’re interested in supporting or collaborating, please get in touch.

💡 Podprite nadaljnji razvoj

Razvijalec trenutno išče finančno podporo za nadaljevanje raziskav in razvoja tega modela ter sorodnih projektov. Za podporo ali sodelovanje vas vabimo, da vzpostavite stik.

📄 Citation

If you use Zlatorog-12B-Instruct-Beta in your research or applications, please cite it as follows:

BibTeX

@misc{zlatorog12b2025,
  title        = {Zlatorog-12B-Instruct-Beta: A Slovenian Instruction-Tuned Large Language Model},
  author       = {Tomaž Savodnik, Zavod za informacijsko družbo (zID)},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/zID4si/Zlatorog-12B-Instruct-Beta}},
  note         = {Version Beta}
}

APA

Savodnik, T., Zavod za informacijsko družbo (zID). (2025). Zlatorog-12B-Instruct-Beta: A Slovenian Instruction-Tuned Large Language Model [Computer software]. Hugging Face. https://huggingface.co/zID4si/Zlatorog-12B-Instruct-Beta

zID4si
/

Zlatorog-12B-Instruct-Beta

Acknowledge license to accept the repository