---
license: mit
tags:
  - svector
  - theta-35-mini
  - theta
---

# Theta-35-mini

A distilled, lightweight version of our Theta-35 main model, built on the Qwen architecture and distilled with the GRPO technique for high efficiency and strong performance in a compact footprint.

## Model Description

**Theta-35-mini** is a small-footprint autoregressive language model distilled from our flagship Theta-35 model. We leveraged:

- **Qwen Model Architecture**: Starting from the Qwen2 base, adapting its efficient transformer blocks and optimized attention kernels.
- **GRPO Distillation**: Guided Representation Projection Optimization (GRPO) to transfer knowledge from Theta-35 to Theta-35-mini, preserving accuracy while drastically reducing parameter count.

This makes Theta-35-mini ideal for on-device inference, low-latency applications, and scenarios with tight compute or memory budgets.

## Intended Uses

- **On-device text generation** (mobile apps, embedded systems)  
- **Real-time chatbots** and conversational agents  
- **Edge AI** applications with strict resource constraints

## Usage

```bash
# Install transformers
pip install transformers

# Load the model
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("SVECTOR-CORPORATION/Theta-35-Mini")
model = AutoModelForCausalLM.from_pretrained("SVECTOR-CORPORATION/Theta-35-Mini")

# Generate text
inputs = tokenizer("Once upon a time", return_tensors="pt")
outputs = model.generate(**inputs, max_length=100, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))