Quantization made by Richard Erkhov.

Yakult-70B - GGUF

Model creator: https://huggingface.co/jan-hq/
Original model: https://huggingface.co/jan-hq/Yakult-70B/

Name	Quant method	Size
Yakult-70B.Q2_K.gguf	Q2_K	24.56GB
Yakult-70B.IQ3_XS.gguf	IQ3_XS	27.29GB
Yakult-70B.IQ3_S.gguf	IQ3_S	28.79GB
Yakult-70B.Q3_K_S.gguf	Q3_K_S	28.79GB
Yakult-70B.IQ3_M.gguf	IQ3_M	29.74GB
Yakult-70B.Q3_K.gguf	Q3_K	31.91GB
Yakult-70B.Q3_K_M.gguf	Q3_K_M	31.91GB
Yakult-70B.Q3_K_L.gguf	Q3_K_L	34.59GB
Yakult-70B.IQ4_XS.gguf	IQ4_XS	35.64GB
Yakult-70B.Q4_0.gguf	Q4_0	37.22GB
Yakult-70B.IQ4_NL.gguf	IQ4_NL	37.58GB
Yakult-70B.Q4_K_S.gguf	Q4_K_S	37.58GB
Yakult-70B.Q4_K.gguf	Q4_K	39.6GB
Yakult-70B.Q4_K_M.gguf	Q4_K_M	39.6GB
Yakult-70B.Q4_1.gguf	Q4_1	41.27GB
Yakult-70B.Q5_0.gguf	Q5_0	45.32GB
Yakult-70B.Q5_K_S.gguf	Q5_K_S	45.32GB
Yakult-70B.Q5_K.gguf	Q5_K	46.52GB
Yakult-70B.Q5_K_M.gguf	Q5_K_M	46.52GB
Yakult-70B.Q5_1.gguf	Q5_1	49.36GB
Yakult-70B.Q6_K.gguf	Q6_K	53.91GB
Yakult-70B.Q8_0.gguf	Q8_0	69.83GB

Original model description:

language: - en license: llama2

Jan - Discord

This is a math fine-tuning of the LLaMA-3 70B model.

Prompt template

Llama3

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{{ system_prompt }}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{ user_message_1 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{{ model_answer_1 }}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Run this model

You can run this model using Jan Desktop on Mac, Windows, or Linux.

Jan is an open source, ChatGPT alternative that is:

💻 100% offline on your machine: Your conversations remain confidential, and visible only to you.
🗂️ ** An Open File Format**: Conversations and model settings stay on your computer and can be exported or deleted at any time.
🌐 OpenAI Compatible: Local server on port 1337 with OpenAI compatible endpoints
🌍 Open Source & Free: We build in public; check out our Github

About Jan

Jan believes in the need for an open-source AI ecosystem and is building the infra and tooling to allow open-source AIs to compete on a level playing field with proprietary ones.

Jan's long-term vision is to build a cognitive framework for future robots, who are practical, useful assistants for humans and businesses in everyday life.