---
license: mit
language:
- vi
pipeline_tag: text-generation
---

# TheSyx-V1-7B-Base

## Introduction

TheSyx-V1-7B-Base is the first LLM released by [thehosy](https://huggingface.co/thehosy).

**Features**:
- Type: Causal Language Models
- Training Stage: Pretraining 
- Architecture: Qwen3 MoE
- Number of Parameters: 7.52B
- Number of Layers: 28
- Number of Attention Heads (GQA): 24 for Q and 4 for KV
- Context Length: Full 16384 tokens and generation 8192 tokens

**We do not recommend using base language models for conversations**. Instead, you can apply post-training, e.g., SFT, RLHF, continued pretraining, etc., on this model.

## Requirements

The code of TheSyx-V1-7B-Base has been in the latest Hugging face `transformers` and i advise you to use the latest version of `transformers`.

## Citation

...