moxin-org/Moxin-7B-Reasoning

Moxin 7B Reasoning

Home Page    |    Technical Report    |    Base Model    |    Chat Model    |    Instruct Model    |    Reasoning Model    |    VLM Model

Chat Template

The chat template is formatted as:

<|system|>\nYou are a helpful AI assistant!\n<|user|>\nHow are you doing?\n<|assistant|>\nThank you for asking! As an AI, I don't have feelings, but I'm functioning normally and ready to assist you. How can I help you today?<|endoftext|>

Or with new lines expanded:

<|system|>
You are a helpful AI assistant!
<|user|>
How are you doing?
<|assistant|>
Thank you for asking! As an AI, I don't have feelings, but I'm functioning normally and ready to assist you. How can I help you today?<|endoftext|>

Inference

You can use the following code to run inference with the model.

import transformers
import torch

model_id = "moxin-org/Moxin-7B-Reasoning"
pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful AI assistant!"},
    {"role": "user", "content": "How are you doing?"},
]

outputs = pipeline(
    messages,
    max_new_tokens=1024,
)

print(outputs[0]["generated_text"][-1])