Upload quantized model

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tekken.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,230 @@
+---
+language:
+- en
+- fr
+- de
+- es
+- pt
+- it
+- ja
+- ko
+- ru
+- zh
+- ar
+- fa
+- id
+- ms
+- ne
+- pl
+- ro
+- sr
+- sv
+- tr
+- uk
+- vi
+- hi
+- bn
+license: apache-2.0
+library_name: vllm
+inference: false
+base_model:
+- mistralai/Mistral-Small-3.1-24B-Instruct-2503
+extra_gated_description: >-
+  If you want to learn more about how we process your personal data, please read
+  our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
+---
+
+# Model Card for Magistral-Small-2506
+
+Building upon Mistral Small 3.1 (2503), **with added reasoning capabilities**, undergoing SFT from Magistral Medium traces and RL on top, it's a small, efficient reasoning model with 24B parameters.
+
+Magistral Small can be deployed locally, fitting within a single RTX 4090 or a 32GB RAM MacBook once quantized.
+
+Learn more about Magistral in our [blog post](https://mistral.ai/news/magistral/).
+
+## Key Features
+- **Reasoning:** Capable of long chains of reasoning traces before providing an answer.
+- **Multilingual:** Supports dozens of languages, including English, French, German, Greek, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Nepali, Polish, Portuguese, Romanian, Russian, Serbian, Spanish, Swedish, Turkish, Ukrainian, Vietnamese, Arabic, Bengali, Chinese, and Farsi.
+- **Apache 2.0 License:** Open license allowing usage and modification for both commercial and non-commercial purposes.
+- **Context Window:** A 128k context window, **but** performance might degrade past **40k**. Hence we recommend setting the maximum model length to 40k.
+
+## Benchmark Results
+
+| Model | AIME24 pass@1 | AIME25 pass@1 | GPQA Diamond | Livecodebench (v5) |
+|-------|-------------|-------------|--------------|-------------------|
+| Magistral Medium | 73.59% | 64.95% | 70.83% | 59.36% |
+| Magistral Small | 70.68% | 62.76% | 68.18% | 55.84% |
+
+## Sampling parameters
+
+Please make sure to use: 
+- `top_p`: 0.95
+- `temperature`: 0.7
+- `max_tokens`: 40960
+
+## Basic Chat Template
+
+We highly recommend including the default system prompt used during RL for the best results, you can edit and customise it if needed for your specific use case.
+
+```
+<s>[SYSTEM_PROMPT]system_prompt
+
+A user will ask you to solve a task. You should first draft your thinking process (inner monologue) until you have derived the final answer. Afterwards, write a self-contained summary of your thoughts (i.e. your summary should be succinct but contain all the critical steps you needed to reach the conclusion). You should use Markdown to format your response. Write both your thoughts and summary in the same language as the task posed by the user. NEVER use \boxed{} in your response.
+
+Your thinking process must follow the template below:
+<think>
+Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate a correct answer.
+</think>
+
+Here, provide a concise summary that reflects your reasoning and presents a clear final answer to the user. Don't mention that this is a summary.
+
+Problem:
+
+[/SYSTEM_PROMPT][INST]user_message[/INST]<think>
+reasoning_traces
+</think>
+assistant_response</s>[INST]user_message[/INST]
+```
+*`system_prompt`, `user_message` and `assistant_response` are placeholders.*
+
+We invite you to choose, depending on your use case and requirements, between keeping reasoning traces during multi-turn interactions or keeping only the final assistant response.
+
+***Please make sure to use [mistral-common](https://github.com/mistralai/mistral-common) as the source of truth***
+
+## Usage
+
+The model can be used with the following frameworks;
+
+### Inference
+
+- [`vllm (recommended)`](https://github.com/vllm-project/vllm): See [below](#vllm)
+
+In addition the community has prepared quantized versions of the model that can be used with the following frameworks (*alphabetically sorted*):
+- [`llama.cpp`](https://github.com/ggml-org/llama.cpp): https://huggingface.co/mistralai/Magistral-Small-2506_gguf
+- [`lmstudio` (llama.cpp, MLX)](https://lmstudio.ai/): https://lmstudio.ai/models/mistralai/magistral-small
+- [`ollama`](https://ollama.com/): https://ollama.com/library/magistral
+- [`unsloth` (llama.cpp)](https://huggingface.co/unsloth): https://huggingface.co/unsloth/Magistral-Small-2506-GGUF
+
+### Training
+
+Fine-tuning is possible with (*alphabetically sorted*):
+- [`axolotl`](https://github.com/axolotl-ai-cloud/axolotl): https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/magistral
+- [`unsloth`](https://github.com/unslothai/unsloth): https://docs.unsloth.ai/basics/magistral
+
+### Other
+
+Also you can use Magistral with:
+- [`kaggle`](https://www.kaggle.com/models/mistral-ai/magistral-small-2506): https://www.kaggle.com/models/mistral-ai/magistral-small-2506
+
+### vLLM (recommended)
+
+We recommend using this model with the [vLLM library](https://github.com/vllm-project/vllm)
+to implement production-ready inference pipelines.
+
+**_Installation_**
+
+Make sure you install the latest [`vLLM`](https://github.com/vllm-project/vllm/) code:
+
+```
+pip install -U vllm \
+    --pre \
+    --extra-index-url https://wheels.vllm.ai/nightly
+```
+
+Doing so should automatically install [`mistral_common >= 1.6.0`](https://github.com/mistralai/mistral-common/releases/tag/v1.6.0).
+
+To check:
+```
+python -c "import mistral_common; print(mistral_common.__version__)"
+```
+
+You can also make use of a ready-to-go [docker image](https://github.com/vllm-project/vllm/blob/main/Dockerfile) or on the [docker hub](https://hub.docker.com/layers/vllm/vllm-openai/latest/images/sha256-de9032a92ffea7b5c007dad80b38fd44aac11eddc31c435f8e52f3b7404bbf39).
+
+
+Serve model as follows:
+
+```
+vllm serve mistralai/Magistral-Small-2506 --tokenizer_mode mistral --config_format mistral --load_format mistral --tool-call-parser mistral --enable-auto-tool-choice --tensor-parallel-size 2
+```
+
+Ping model as follows:
+
+```py
+from openai import OpenAI
+from huggingface_hub import hf_hub_download
+
+# Modify OpenAI's API key and API base to use vLLM's API server.
+openai_api_key = "EMPTY"
+openai_api_base = "http://localhost:8000/v1"
+
+TEMP = 0.7
+TOP_P = 0.95
+MAX_TOK = 40_960
+
+client = OpenAI(
+    api_key=openai_api_key,
+    base_url=openai_api_base,
+)
+
+models = client.models.list()
+model = models.data[0].id
+
+def load_system_prompt(repo_id: str, filename: str) -> str:
+    file_path = hf_hub_download(repo_id=repo_id, filename=filename)
+    with open(file_path, "r") as file:
+        system_prompt = file.read()
+    return system_prompt
+
+SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
+
+query = "Write 4 sentences, each with at least 8 words. Now make absolutely sure that every sentence has exactly one word less than the previous sentence."
+# or try out other queries
+# query = "Exactly how many days ago did the French Revolution start? Today is June 4th, 2025."
+# query = "Think about 5 random numbers. Verify if you can combine them with addition, multiplication, subtraction or division to 133"
+# query = "If it takes 30 minutes to dry 12 T-shirts in the sun, how long does it take to dry 33 T-shirts?"
+
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {"role": "user", "content": query}
+]
+stream = client.chat.completions.create(
+  model=model,
+  messages=messages,
+  stream=True,
+  temperature=TEMP,
+  top_p=TOP_P,
+  max_tokens=MAX_TOK,
+)
+
+print("client: Start streaming chat completions...")
+printed_content = False
+
+for chunk in stream:
+  content = None
+  # Check the content is content
+  if hasattr(chunk.choices[0].delta, "content"):
+    content = chunk.choices[0].delta.content
+
+  if content is not None:
+    if not printed_content:
+        printed_content = True
+        print("\ncontent:", end="", flush=True)
+    # Extract and print the content
+    print(content, end="", flush=True)
+
+# content:<think>
+# Alright, I need to write 4 sentences where each one has at least 8 words and each subsequent sentence has one fewer word than the previous one.
+# ...
+# Final boxed answer (the four sentences):
+
+# \[
+# \boxed{
+# \begin{aligned}
+# &\text{1. The quick brown fox jumps over lazy dog and yells hello.} \\
+# &\text{2. I saw the cat on the stair with my hat.} \\
+# &\text{3. The man in the moon came down quickly today.} \\
+# &\text{4. A cat sat on the mat today patiently.}
+# \end{aligned}
+# }
+# \]
+```

SYSTEM_PROMPT.txt

@@ -0,0 +1,10 @@
+A user will ask you to solve a task. You should first draft your thinking process (inner monologue) until you have derived the final answer. Afterwards, write a self-contained summary of your thoughts (i.e. your summary should be succinct but contain all the critical steps you needed to reach the conclusion). You should use Markdown and Latex to format your response. Write both your thoughts and summary in the same language as the task posed by the user.
+
+Your thinking process must follow the template below:
+<think>
+Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate a correct answer.
+</think>
+
+Here, provide a concise summary that reflects your reasoning and presents a clear final answer to the user.
+
+Problem:

config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 32768,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 8,
+  "quantization_config": {
+    "bits": 4,
+    "group_size": 128,
+    "modules_to_not_convert": null,
+    "quant_method": "awq",
+    "version": "gemm",
+    "zero_point": true
+  },
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 1000000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.53.0.dev0",
+  "use_cache": false,
+  "vocab_size": 131072
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": 2,
+  "transformers_version": "4.53.0.dev0"
+}

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbc1bcb2e79603d2835d0780464218d4bf5343011d4a96efc97e30a3299c87ba
+size 4921581912

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:532897176a56dcf2570e9c800b78d9c86b15622bd7452b5a422da917c91e3995
+size 4995915728

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c965633b909ff5b00de924d3bbd5959f9cba4353c57d29292a5abed09b1ba0e
+size 4316873088

model.safetensors.index.json

@@ -0,0 +1,930 @@
+{
+  "metadata": {
+    "total_size": 14234265600
+  },
+  "weight_map": {
+    "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.11.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.up_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.up_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.up_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.down_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.down_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.11.mlp.down_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.q_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.q_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.q_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.k_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.k_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.k_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.v_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.v_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.v_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.o_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.o_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.12.self_attn.o_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.12.mlp.gate_proj.qweight": "model-00001-of-00003.safetensors",
+    "model.layers.12.mlp.gate_proj.qzeros": "model-00001-of-00003.safetensors",
+    "model.layers.12.mlp.gate_proj.scales": "model-00001-of-00003.safetensors",
+    "model.layers.12.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.22.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.23.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.23.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.24.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.24.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.25.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.25.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.26.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.26.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.27.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.27.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.27.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.27.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.27.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.27.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.27.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.27.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.27.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.27.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.28.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.28.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.28.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.28.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.28.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.28.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.28.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.28.mlp.down_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.28.mlp.down_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.28.mlp.down_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.q_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.q_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.q_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.k_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.k_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.k_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.v_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.v_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.v_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.o_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.o_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.29.self_attn.o_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.29.mlp.gate_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.29.mlp.gate_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.29.mlp.gate_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.29.mlp.up_proj.qweight": "model-00002-of-00003.safetensors",
+    "model.layers.29.mlp.up_proj.qzeros": "model-00002-of-00003.safetensors",
+    "model.layers.29.mlp.up_proj.scales": "model-00002-of-00003.safetensors",
+    "model.layers.29.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.32.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.32.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.32.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.32.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.32.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.32.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.32.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.32.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.32.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.32.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.33.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.33.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.33.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.33.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.33.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.33.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.33.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.33.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.33.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.33.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.34.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.34.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.34.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.34.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.34.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.34.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.34.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.34.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.34.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.34.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.35.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.35.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.35.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.35.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.35.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.35.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.35.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.35.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.35.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.35.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.36.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.36.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.36.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.36.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.36.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.36.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.36.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.36.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.36.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.36.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.36.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.36.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.37.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.37.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.37.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.37.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.37.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.37.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.37.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.37.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.37.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.37.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.37.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.37.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.38.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.38.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.38.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.38.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.38.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.38.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.38.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.38.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.38.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.38.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.38.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.38.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.q_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.q_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.q_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.k_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.k_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.k_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.v_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.v_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.v_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.o_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.o_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.39.self_attn.o_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.39.mlp.gate_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.39.mlp.gate_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.39.mlp.gate_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.39.mlp.up_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.39.mlp.up_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.39.mlp.up_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.39.mlp.down_proj.qweight": "model-00003-of-00003.safetensors",
+    "model.layers.39.mlp.down_proj.qzeros": "model-00003-of-00003.safetensors",
+    "model.layers.39.mlp.down_proj.scales": "model-00003-of-00003.safetensors",
+    "model.layers.39.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.39.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.norm.weight": "model-00003-of-00003.safetensors",
+    "lm_head.weight": "model-00003-of-00003.safetensors"
+  }
+}

params.json

@@ -0,0 +1,11 @@
+{
+  "dim": 5120,
+  "n_layers": 40,
+  "head_dim": 128,
+  "hidden_dim": 32768,
+  "n_heads": 32,
+  "n_kv_heads": 8,
+  "rope_theta": 1000000000.0,
+  "norm_eps": 1e-05,
+  "vocab_size": 131072
+}

tekken.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85a515798166e759a4cded03b01373dd6bf3c4e801c4e03a989169ddc09cac08
+size 19399778