Upload model

README.md

@@ -1,77 +1,3 @@
----
-license: apache-2.0
----
-
-## 📖 Introduction
-
-# DistilQwen2.5-DS3-0324 Series: Fast-Thinking Reasoning Models
-
-## Overview
-In response to the industry challenge of balancing efficient reasoning with cognitive capabilities, the DistilQwen2.5-DS3-0324 series innovatively transfers the fast-thinking capabilities of DeepSeekV3-0324 to lightweight models. Through a two-stage distillation framework, this series achieves high performance while delivering:
-- **Enhanced Reasoning Speed**: Reduces output tokens by 60-80% (compared to slow-thinking models)
-- **Reduced Resource Consumption**: Suitable for edge computing deployment
-- **Elimination of Cognitive Bias**: Proprietary trajectory alignment technology
-
-## Core Innovations
-### 1. Fast-Thinking Distillation Framework
-- **Stage 1: Fast-Thinking CoT Data Collection**
-  - **Long-to-Short Rewriting**: Extracts key reasoning steps from DeepSeek-R1
-  - **Teacher Model Distillation**: Captures the rapid reasoning trajectories of DeepSeekV3-0324
-
-- **Stage 2: CoT Trajectory Cognitive Alignment**
-  - **Dynamic Difficulty Grading** (Easy/Medium/Hard)
-    - LLM-as-a-Judge evaluates small model comprehensibility
-    - Simple chain expansion → Adds necessary steps
-    - Hard chain simplification → Removes high-level logical leaps
-  - **Validation Mechanism**: Iterative optimization until all data reaches "Medium" rating
-
-### 2. Performance Breakthroughs
-- **32B Model** approaches the performance of closed-source models with 10x the parameters on the GPQA Diamond benchmark
-- **Significant Improvement in Reasoning Efficiency** (see comparison table below)
-
-| Model                          | MMLU_PRO Tokens | AIME2024 Tokens | Speed Gain |
-|--------------------------------|-----------------|-----------------|------------|
-| DistilQwen2.5-R1-32B (Slow-Thinking) | 4198            | 12178           | 1x         |
-| DistilQwen2.5-DS3-0324-32B     | 690             | 4177            | 5-8x       |
-
-## Technical Advantages
-- **Two-Stage Distillation**: First compresses reasoning length, then aligns cognitive trajectories
-- **Dynamic Data Optimization**: Adaptive difficulty adjustment ensures knowledge transferability
-- **Open-Source Compatibility**: Fine-tuned based on the Qwen2.5 base model
-
-## 🚀 Quick Start
-
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-device = "cuda" # the device to load the model onto
-
-model = AutoModelForCausalLM.from_pretrained(
-    "alibaba-pai/DistilQwen2.5-DS3-0324-32B",
-    torch_dtype="auto",
-    device_map="auto"
-)
-tokenizer = AutoTokenizer.from_pretrained("alibaba-pai/DistilQwen2.5-DS3-0324-32B")
-
-prompt = "Give me a short introduction to large language model."
-messages=[
-    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant. You should think step-by-step."},
-    {"role": "user", "content": prompt},
-]
-text = tokenizer.apply_chat_template(
-    messages,
-    tokenize=False,
-    add_generation_prompt=True
-)
-model_inputs = tokenizer([text], return_tensors="pt").to(device)
-
-generated_ids = model.generate(
-    model_inputs.input_ids,
-    max_new_tokens=2048，
-)
-generated_ids = [
-    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
-]
-
-response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
-
-```
+---
+license: apache-2.0
+---

config.json

@@ -0,0 +1,29 @@
+{
+  "_name_or_path": "/mnt/workspace/DistilQwen25-DS3-0324/32B",
+  "architectures": [
+    "Qwen2Model"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 27648,
+  "max_position_embeddings": 32768,
+  "max_window_layers": 70,
+  "model_type": "qwen2",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 64,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.3",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 152064
+}

model-00001-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7c226a8aac120d2d505dd6b47488a9033948c906615e6bb870e84cc6906024b
+size 4498420808

model-00002-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69d5dd89425d85746c12f59c6f735ed399fe4b3b7cac0812abe97fcfa03214b0
+size 4718804568

model-00003-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:316ca317a362abe7b3a358524e06881e209c8eaabc1c6231357484355ae37a6e
+size 4467075728

model-00004-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f9e0b6e1512ac1d83ed39236cb719b91d067a7587f13e646aaa68183668ea72
+size 4467075728

model-00005-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e095f48ae49995e56b8a29dd5254bed925ace67ee3d8336b07fb48b0fa9dd3dc
+size 4718804552

model-00006-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:258da1e07d37bb8b05f79dbb58a11648bd2fc59ba2a86ae8080db6bd81c0498d
+size 4467075752

model-00007-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b6651cb5a7b89caba34a0eda69ccf8081eb41343f2b49468a50b85990e36993
+size 4467075752

model-00008-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd2b2a73b0e8a64226a565de4d36c445857f4cd1b2603c3b367741b6356c6741
+size 4718804600

model-00009-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f010ab66254e4c9d5798aa19e08b4305d7b99a6169219a671d6c2b39c8b8a20
+size 4467075752

model-00010-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe532cf584db628253d42405c3a3ca94d3e583c2e2b1f910bfd94e7e0a4f7548
+size 4467075752

model-00011-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0fca493c0ca4b827c9caa1b167cffc21ae2830eecd98e51e85fb06eb7080cd38
+size 4718804600

model-00012-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87eed171cf2553b883416a4b0b81d2b037e729f87ec5c919e9fd90582c3545be
+size 4467075752

model-00013-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11eb952c3174c60651a6becd03fbdefdd7e7ce1f7f60fc11be4dd7bee74f7eb0
+size 4467075752

model-00014-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f4a666a776f4e1ada88d49ad0183fb5edfe48640c4344483f0db0eacff8b420
+size 4718804600

model-00015-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90a0bdf4b66507ebe1db9e208213b4e79616871106973ef2de7c29563a6f2c14
+size 4467075752

model-00016-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5083f28647e8d2d0cb1c99bb79a73c88dd98e61aedf5ed8343d5e71031d4778
+size 4467075752

model-00017-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e11a243810dc2e10833a30e40e0e799c26b0b40bfb6c8c23ab3f187d42d5213c
+size 4718804600

model-00018-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:617e994d7a90a21f45025e2dc5f3a9d5d3f2e54ad84a1896c35892f0cd8d4ee9
+size 4467075752

model-00019-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b679dca53ba59d8caf10c00f32a3c6355457befd910fecd09853c2e97e8127f5
+size 4467075752

model-00020-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d8f6bdd2918c71aef07da4a02ba768db6333085d0945b034dab8f36ab8e4dd4
+size 4718804600

model-00021-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae18ec1e155bbc915af1be920cdac5fb388710e57a9d47794d99f3c1ec4baf86
+size 4467075752

model-00022-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a210ebeed8d1a62a4e840ed4c91115b5583c4081f5da7131cceb5020cbd4ab5
+size 4467075752

model-00023-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a685daa3a23c694731b8bdab02ae1c9a579e7367aeb3b2b05645fb3a4c1e7c81
+size 4718804600

model-00024-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b6a1cba16f9a0d650a44288827c2303d8dfd18e1f24ab21a55a677351d91658
+size 4467075752

model-00025-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6595d08fa06f038c631e34b815296359f0c819c3e4d5d7f54cca990f31f64298
+size 4467075752

model-00026-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bb8b19ec241fdae571d05258b29f9c0daac14708cccaf0b56b24ea359ee1496
+size 4718804600

model-00027-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ccb260c87ab79475c370134604b9e6e8c1037fd04e6851642480afe737dc25f
+size 4467075752

model-00028-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b504c6fcb677ea4c4fbe0be1e596b420a045137dd09fdf00429cee5725bcf8d
+size 4467075752

model-00029-of-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df07e8030b67d2001c5696934a45973a9fbe6935ad5fed89c8be185da813beaa
+size 566292912

model.safetensors.index.json

@@ -0,0 +1,777 @@
+{
+  "metadata": {
+    "total_size": 127941234688
+  },
+  "weight_map": {
+    "embed_tokens.weight": "model-00001-of-00029.safetensors",
+    "layers.0.input_layernorm.weight": "model-00002-of-00029.safetensors",
+    "layers.0.mlp.down_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.0.mlp.gate_proj.weight": "model-00001-of-00029.safetensors",
+    "layers.0.mlp.up_proj.weight": "model-00001-of-00029.safetensors",
+    "layers.0.post_attention_layernorm.weight": "model-00002-of-00029.safetensors",
+    "layers.0.self_attn.k_proj.bias": "model-00001-of-00029.safetensors",
+    "layers.0.self_attn.k_proj.weight": "model-00001-of-00029.safetensors",
+    "layers.0.self_attn.o_proj.weight": "model-00001-of-00029.safetensors",
+    "layers.0.self_attn.q_proj.bias": "model-00001-of-00029.safetensors",
+    "layers.0.self_attn.q_proj.weight": "model-00001-of-00029.safetensors",
+    "layers.0.self_attn.v_proj.bias": "model-00001-of-00029.safetensors",
+    "layers.0.self_attn.v_proj.weight": "model-00001-of-00029.safetensors",
+    "layers.1.input_layernorm.weight": "model-00002-of-00029.safetensors",
+    "layers.1.mlp.down_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.1.mlp.gate_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.1.mlp.up_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.1.post_attention_layernorm.weight": "model-00002-of-00029.safetensors",
+    "layers.1.self_attn.k_proj.bias": "model-00002-of-00029.safetensors",
+    "layers.1.self_attn.k_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.1.self_attn.o_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.1.self_attn.q_proj.bias": "model-00002-of-00029.safetensors",
+    "layers.1.self_attn.q_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.1.self_attn.v_proj.bias": "model-00002-of-00029.safetensors",
+    "layers.1.self_attn.v_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.10.input_layernorm.weight": "model-00006-of-00029.safetensors",
+    "layers.10.mlp.down_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.10.mlp.gate_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.10.mlp.up_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.10.post_attention_layernorm.weight": "model-00006-of-00029.safetensors",
+    "layers.10.self_attn.k_proj.bias": "model-00005-of-00029.safetensors",
+    "layers.10.self_attn.k_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.10.self_attn.o_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.10.self_attn.q_proj.bias": "model-00005-of-00029.safetensors",
+    "layers.10.self_attn.q_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.10.self_attn.v_proj.bias": "model-00005-of-00029.safetensors",
+    "layers.10.self_attn.v_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.11.input_layernorm.weight": "model-00006-of-00029.safetensors",
+    "layers.11.mlp.down_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.11.mlp.gate_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.11.mlp.up_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.11.post_attention_layernorm.weight": "model-00006-of-00029.safetensors",
+    "layers.11.self_attn.k_proj.bias": "model-00006-of-00029.safetensors",
+    "layers.11.self_attn.k_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.11.self_attn.o_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.11.self_attn.q_proj.bias": "model-00006-of-00029.safetensors",
+    "layers.11.self_attn.q_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.11.self_attn.v_proj.bias": "model-00006-of-00029.safetensors",
+    "layers.11.self_attn.v_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.12.input_layernorm.weight": "model-00007-of-00029.safetensors",
+    "layers.12.mlp.down_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.12.mlp.gate_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.12.mlp.up_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.12.post_attention_layernorm.weight": "model-00007-of-00029.safetensors",
+    "layers.12.self_attn.k_proj.bias": "model-00006-of-00029.safetensors",
+    "layers.12.self_attn.k_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.12.self_attn.o_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.12.self_attn.q_proj.bias": "model-00006-of-00029.safetensors",
+    "layers.12.self_attn.q_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.12.self_attn.v_proj.bias": "model-00006-of-00029.safetensors",
+    "layers.12.self_attn.v_proj.weight": "model-00006-of-00029.safetensors",
+    "layers.13.input_layernorm.weight": "model-00007-of-00029.safetensors",
+    "layers.13.mlp.down_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.13.mlp.gate_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.13.mlp.up_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.13.post_attention_layernorm.weight": "model-00007-of-00029.safetensors",
+    "layers.13.self_attn.k_proj.bias": "model-00007-of-00029.safetensors",
+    "layers.13.self_attn.k_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.13.self_attn.o_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.13.self_attn.q_proj.bias": "model-00007-of-00029.safetensors",
+    "layers.13.self_attn.q_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.13.self_attn.v_proj.bias": "model-00007-of-00029.safetensors",
+    "layers.13.self_attn.v_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.14.input_layernorm.weight": "model-00008-of-00029.safetensors",
+    "layers.14.mlp.down_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.14.mlp.gate_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.14.mlp.up_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.14.post_attention_layernorm.weight": "model-00008-of-00029.safetensors",
+    "layers.14.self_attn.k_proj.bias": "model-00007-of-00029.safetensors",
+    "layers.14.self_attn.k_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.14.self_attn.o_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.14.self_attn.q_proj.bias": "model-00007-of-00029.safetensors",
+    "layers.14.self_attn.q_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.14.self_attn.v_proj.bias": "model-00007-of-00029.safetensors",
+    "layers.14.self_attn.v_proj.weight": "model-00007-of-00029.safetensors",
+    "layers.15.input_layernorm.weight": "model-00008-of-00029.safetensors",
+    "layers.15.mlp.down_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.15.mlp.gate_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.15.mlp.up_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.15.post_attention_layernorm.weight": "model-00008-of-00029.safetensors",
+    "layers.15.self_attn.k_proj.bias": "model-00008-of-00029.safetensors",
+    "layers.15.self_attn.k_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.15.self_attn.o_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.15.self_attn.q_proj.bias": "model-00008-of-00029.safetensors",
+    "layers.15.self_attn.q_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.15.self_attn.v_proj.bias": "model-00008-of-00029.safetensors",
+    "layers.15.self_attn.v_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.16.input_layernorm.weight": "model-00008-of-00029.safetensors",
+    "layers.16.mlp.down_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.16.mlp.gate_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.16.mlp.up_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.16.post_attention_layernorm.weight": "model-00008-of-00029.safetensors",
+    "layers.16.self_attn.k_proj.bias": "model-00008-of-00029.safetensors",
+    "layers.16.self_attn.k_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.16.self_attn.o_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.16.self_attn.q_proj.bias": "model-00008-of-00029.safetensors",
+    "layers.16.self_attn.q_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.16.self_attn.v_proj.bias": "model-00008-of-00029.safetensors",
+    "layers.16.self_attn.v_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.17.input_layernorm.weight": "model-00009-of-00029.safetensors",
+    "layers.17.mlp.down_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.17.mlp.gate_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.17.mlp.up_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.17.post_attention_layernorm.weight": "model-00009-of-00029.safetensors",
+    "layers.17.self_attn.k_proj.bias": "model-00008-of-00029.safetensors",
+    "layers.17.self_attn.k_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.17.self_attn.o_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.17.self_attn.q_proj.bias": "model-00008-of-00029.safetensors",
+    "layers.17.self_attn.q_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.17.self_attn.v_proj.bias": "model-00008-of-00029.safetensors",
+    "layers.17.self_attn.v_proj.weight": "model-00008-of-00029.safetensors",
+    "layers.18.input_layernorm.weight": "model-00009-of-00029.safetensors",
+    "layers.18.mlp.down_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.18.mlp.gate_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.18.mlp.up_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.18.post_attention_layernorm.weight": "model-00009-of-00029.safetensors",
+    "layers.18.self_attn.k_proj.bias": "model-00009-of-00029.safetensors",
+    "layers.18.self_attn.k_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.18.self_attn.o_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.18.self_attn.q_proj.bias": "model-00009-of-00029.safetensors",
+    "layers.18.self_attn.q_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.18.self_attn.v_proj.bias": "model-00009-of-00029.safetensors",
+    "layers.18.self_attn.v_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.19.input_layernorm.weight": "model-00010-of-00029.safetensors",
+    "layers.19.mlp.down_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.19.mlp.gate_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.19.mlp.up_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.19.post_attention_layernorm.weight": "model-00010-of-00029.safetensors",
+    "layers.19.self_attn.k_proj.bias": "model-00009-of-00029.safetensors",
+    "layers.19.self_attn.k_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.19.self_attn.o_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.19.self_attn.q_proj.bias": "model-00009-of-00029.safetensors",
+    "layers.19.self_attn.q_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.19.self_attn.v_proj.bias": "model-00009-of-00029.safetensors",
+    "layers.19.self_attn.v_proj.weight": "model-00009-of-00029.safetensors",
+    "layers.2.input_layernorm.weight": "model-00002-of-00029.safetensors",
+    "layers.2.mlp.down_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.2.mlp.gate_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.2.mlp.up_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.2.post_attention_layernorm.weight": "model-00002-of-00029.safetensors",
+    "layers.2.self_attn.k_proj.bias": "model-00002-of-00029.safetensors",
+    "layers.2.self_attn.k_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.2.self_attn.o_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.2.self_attn.q_proj.bias": "model-00002-of-00029.safetensors",
+    "layers.2.self_attn.q_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.2.self_attn.v_proj.bias": "model-00002-of-00029.safetensors",
+    "layers.2.self_attn.v_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.20.input_layernorm.weight": "model-00010-of-00029.safetensors",
+    "layers.20.mlp.down_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.20.mlp.gate_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.20.mlp.up_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.20.post_attention_layernorm.weight": "model-00010-of-00029.safetensors",
+    "layers.20.self_attn.k_proj.bias": "model-00010-of-00029.safetensors",
+    "layers.20.self_attn.k_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.20.self_attn.o_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.20.self_attn.q_proj.bias": "model-00010-of-00029.safetensors",
+    "layers.20.self_attn.q_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.20.self_attn.v_proj.bias": "model-00010-of-00029.safetensors",
+    "layers.20.self_attn.v_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.21.input_layernorm.weight": "model-00011-of-00029.safetensors",
+    "layers.21.mlp.down_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.21.mlp.gate_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.21.mlp.up_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.21.post_attention_layernorm.weight": "model-00011-of-00029.safetensors",
+    "layers.21.self_attn.k_proj.bias": "model-00010-of-00029.safetensors",
+    "layers.21.self_attn.k_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.21.self_attn.o_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.21.self_attn.q_proj.bias": "model-00010-of-00029.safetensors",
+    "layers.21.self_attn.q_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.21.self_attn.v_proj.bias": "model-00010-of-00029.safetensors",
+    "layers.21.self_attn.v_proj.weight": "model-00010-of-00029.safetensors",
+    "layers.22.input_layernorm.weight": "model-00011-of-00029.safetensors",
+    "layers.22.mlp.down_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.22.mlp.gate_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.22.mlp.up_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.22.post_attention_layernorm.weight": "model-00011-of-00029.safetensors",
+    "layers.22.self_attn.k_proj.bias": "model-00011-of-00029.safetensors",
+    "layers.22.self_attn.k_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.22.self_attn.o_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.22.self_attn.q_proj.bias": "model-00011-of-00029.safetensors",
+    "layers.22.self_attn.q_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.22.self_attn.v_proj.bias": "model-00011-of-00029.safetensors",
+    "layers.22.self_attn.v_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.23.input_layernorm.weight": "model-00011-of-00029.safetensors",
+    "layers.23.mlp.down_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.23.mlp.gate_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.23.mlp.up_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.23.post_attention_layernorm.weight": "model-00011-of-00029.safetensors",
+    "layers.23.self_attn.k_proj.bias": "model-00011-of-00029.safetensors",
+    "layers.23.self_attn.k_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.23.self_attn.o_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.23.self_attn.q_proj.bias": "model-00011-of-00029.safetensors",
+    "layers.23.self_attn.q_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.23.self_attn.v_proj.bias": "model-00011-of-00029.safetensors",
+    "layers.23.self_attn.v_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.24.input_layernorm.weight": "model-00012-of-00029.safetensors",
+    "layers.24.mlp.down_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.24.mlp.gate_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.24.mlp.up_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.24.post_attention_layernorm.weight": "model-00012-of-00029.safetensors",
+    "layers.24.self_attn.k_proj.bias": "model-00011-of-00029.safetensors",
+    "layers.24.self_attn.k_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.24.self_attn.o_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.24.self_attn.q_proj.bias": "model-00011-of-00029.safetensors",
+    "layers.24.self_attn.q_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.24.self_attn.v_proj.bias": "model-00011-of-00029.safetensors",
+    "layers.24.self_attn.v_proj.weight": "model-00011-of-00029.safetensors",
+    "layers.25.input_layernorm.weight": "model-00012-of-00029.safetensors",
+    "layers.25.mlp.down_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.25.mlp.gate_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.25.mlp.up_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.25.post_attention_layernorm.weight": "model-00012-of-00029.safetensors",
+    "layers.25.self_attn.k_proj.bias": "model-00012-of-00029.safetensors",
+    "layers.25.self_attn.k_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.25.self_attn.o_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.25.self_attn.q_proj.bias": "model-00012-of-00029.safetensors",
+    "layers.25.self_attn.q_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.25.self_attn.v_proj.bias": "model-00012-of-00029.safetensors",
+    "layers.25.self_attn.v_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.26.input_layernorm.weight": "model-00013-of-00029.safetensors",
+    "layers.26.mlp.down_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.26.mlp.gate_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.26.mlp.up_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.26.post_attention_layernorm.weight": "model-00013-of-00029.safetensors",
+    "layers.26.self_attn.k_proj.bias": "model-00012-of-00029.safetensors",
+    "layers.26.self_attn.k_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.26.self_attn.o_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.26.self_attn.q_proj.bias": "model-00012-of-00029.safetensors",
+    "layers.26.self_attn.q_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.26.self_attn.v_proj.bias": "model-00012-of-00029.safetensors",
+    "layers.26.self_attn.v_proj.weight": "model-00012-of-00029.safetensors",
+    "layers.27.input_layernorm.weight": "model-00013-of-00029.safetensors",
+    "layers.27.mlp.down_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.27.mlp.gate_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.27.mlp.up_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.27.post_attention_layernorm.weight": "model-00013-of-00029.safetensors",
+    "layers.27.self_attn.k_proj.bias": "model-00013-of-00029.safetensors",
+    "layers.27.self_attn.k_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.27.self_attn.o_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.27.self_attn.q_proj.bias": "model-00013-of-00029.safetensors",
+    "layers.27.self_attn.q_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.27.self_attn.v_proj.bias": "model-00013-of-00029.safetensors",
+    "layers.27.self_attn.v_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.28.input_layernorm.weight": "model-00014-of-00029.safetensors",
+    "layers.28.mlp.down_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.28.mlp.gate_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.28.mlp.up_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.28.post_attention_layernorm.weight": "model-00014-of-00029.safetensors",
+    "layers.28.self_attn.k_proj.bias": "model-00013-of-00029.safetensors",
+    "layers.28.self_attn.k_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.28.self_attn.o_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.28.self_attn.q_proj.bias": "model-00013-of-00029.safetensors",
+    "layers.28.self_attn.q_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.28.self_attn.v_proj.bias": "model-00013-of-00029.safetensors",
+    "layers.28.self_attn.v_proj.weight": "model-00013-of-00029.safetensors",
+    "layers.29.input_layernorm.weight": "model-00014-of-00029.safetensors",
+    "layers.29.mlp.down_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.29.mlp.gate_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.29.mlp.up_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.29.post_attention_layernorm.weight": "model-00014-of-00029.safetensors",
+    "layers.29.self_attn.k_proj.bias": "model-00014-of-00029.safetensors",
+    "layers.29.self_attn.k_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.29.self_attn.o_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.29.self_attn.q_proj.bias": "model-00014-of-00029.safetensors",
+    "layers.29.self_attn.q_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.29.self_attn.v_proj.bias": "model-00014-of-00029.safetensors",
+    "layers.29.self_attn.v_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.3.input_layernorm.weight": "model-00003-of-00029.safetensors",
+    "layers.3.mlp.down_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.3.mlp.gate_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.3.mlp.up_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.3.post_attention_layernorm.weight": "model-00003-of-00029.safetensors",
+    "layers.3.self_attn.k_proj.bias": "model-00002-of-00029.safetensors",
+    "layers.3.self_attn.k_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.3.self_attn.o_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.3.self_attn.q_proj.bias": "model-00002-of-00029.safetensors",
+    "layers.3.self_attn.q_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.3.self_attn.v_proj.bias": "model-00002-of-00029.safetensors",
+    "layers.3.self_attn.v_proj.weight": "model-00002-of-00029.safetensors",
+    "layers.30.input_layernorm.weight": "model-00014-of-00029.safetensors",
+    "layers.30.mlp.down_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.30.mlp.gate_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.30.mlp.up_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.30.post_attention_layernorm.weight": "model-00014-of-00029.safetensors",
+    "layers.30.self_attn.k_proj.bias": "model-00014-of-00029.safetensors",
+    "layers.30.self_attn.k_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.30.self_attn.o_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.30.self_attn.q_proj.bias": "model-00014-of-00029.safetensors",
+    "layers.30.self_attn.q_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.30.self_attn.v_proj.bias": "model-00014-of-00029.safetensors",
+    "layers.30.self_attn.v_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.31.input_layernorm.weight": "model-00015-of-00029.safetensors",
+    "layers.31.mlp.down_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.31.mlp.gate_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.31.mlp.up_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.31.post_attention_layernorm.weight": "model-00015-of-00029.safetensors",
+    "layers.31.self_attn.k_proj.bias": "model-00014-of-00029.safetensors",
+    "layers.31.self_attn.k_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.31.self_attn.o_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.31.self_attn.q_proj.bias": "model-00014-of-00029.safetensors",
+    "layers.31.self_attn.q_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.31.self_attn.v_proj.bias": "model-00014-of-00029.safetensors",
+    "layers.31.self_attn.v_proj.weight": "model-00014-of-00029.safetensors",
+    "layers.32.input_layernorm.weight": "model-00015-of-00029.safetensors",
+    "layers.32.mlp.down_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.32.mlp.gate_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.32.mlp.up_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.32.post_attention_layernorm.weight": "model-00015-of-00029.safetensors",
+    "layers.32.self_attn.k_proj.bias": "model-00015-of-00029.safetensors",
+    "layers.32.self_attn.k_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.32.self_attn.o_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.32.self_attn.q_proj.bias": "model-00015-of-00029.safetensors",
+    "layers.32.self_attn.q_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.32.self_attn.v_proj.bias": "model-00015-of-00029.safetensors",
+    "layers.32.self_attn.v_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.33.input_layernorm.weight": "model-00016-of-00029.safetensors",
+    "layers.33.mlp.down_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.33.mlp.gate_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.33.mlp.up_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.33.post_attention_layernorm.weight": "model-00016-of-00029.safetensors",
+    "layers.33.self_attn.k_proj.bias": "model-00015-of-00029.safetensors",
+    "layers.33.self_attn.k_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.33.self_attn.o_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.33.self_attn.q_proj.bias": "model-00015-of-00029.safetensors",
+    "layers.33.self_attn.q_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.33.self_attn.v_proj.bias": "model-00015-of-00029.safetensors",
+    "layers.33.self_attn.v_proj.weight": "model-00015-of-00029.safetensors",
+    "layers.34.input_layernorm.weight": "model-00016-of-00029.safetensors",
+    "layers.34.mlp.down_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.34.mlp.gate_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.34.mlp.up_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.34.post_attention_layernorm.weight": "model-00016-of-00029.safetensors",
+    "layers.34.self_attn.k_proj.bias": "model-00016-of-00029.safetensors",
+    "layers.34.self_attn.k_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.34.self_attn.o_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.34.self_attn.q_proj.bias": "model-00016-of-00029.safetensors",
+    "layers.34.self_attn.q_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.34.self_attn.v_proj.bias": "model-00016-of-00029.safetensors",
+    "layers.34.self_attn.v_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.35.input_layernorm.weight": "model-00017-of-00029.safetensors",
+    "layers.35.mlp.down_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.35.mlp.gate_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.35.mlp.up_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.35.post_attention_layernorm.weight": "model-00017-of-00029.safetensors",
+    "layers.35.self_attn.k_proj.bias": "model-00016-of-00029.safetensors",
+    "layers.35.self_attn.k_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.35.self_attn.o_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.35.self_attn.q_proj.bias": "model-00016-of-00029.safetensors",
+    "layers.35.self_attn.q_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.35.self_attn.v_proj.bias": "model-00016-of-00029.safetensors",
+    "layers.35.self_attn.v_proj.weight": "model-00016-of-00029.safetensors",
+    "layers.36.input_layernorm.weight": "model-00017-of-00029.safetensors",
+    "layers.36.mlp.down_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.36.mlp.gate_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.36.mlp.up_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.36.post_attention_layernorm.weight": "model-00017-of-00029.safetensors",
+    "layers.36.self_attn.k_proj.bias": "model-00017-of-00029.safetensors",
+    "layers.36.self_attn.k_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.36.self_attn.o_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.36.self_attn.q_proj.bias": "model-00017-of-00029.safetensors",
+    "layers.36.self_attn.q_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.36.self_attn.v_proj.bias": "model-00017-of-00029.safetensors",
+    "layers.36.self_attn.v_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.37.input_layernorm.weight": "model-00017-of-00029.safetensors",
+    "layers.37.mlp.down_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.37.mlp.gate_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.37.mlp.up_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.37.post_attention_layernorm.weight": "model-00017-of-00029.safetensors",
+    "layers.37.self_attn.k_proj.bias": "model-00017-of-00029.safetensors",
+    "layers.37.self_attn.k_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.37.self_attn.o_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.37.self_attn.q_proj.bias": "model-00017-of-00029.safetensors",
+    "layers.37.self_attn.q_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.37.self_attn.v_proj.bias": "model-00017-of-00029.safetensors",
+    "layers.37.self_attn.v_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.38.input_layernorm.weight": "model-00018-of-00029.safetensors",
+    "layers.38.mlp.down_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.38.mlp.gate_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.38.mlp.up_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.38.post_attention_layernorm.weight": "model-00018-of-00029.safetensors",
+    "layers.38.self_attn.k_proj.bias": "model-00017-of-00029.safetensors",
+    "layers.38.self_attn.k_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.38.self_attn.o_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.38.self_attn.q_proj.bias": "model-00017-of-00029.safetensors",
+    "layers.38.self_attn.q_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.38.self_attn.v_proj.bias": "model-00017-of-00029.safetensors",
+    "layers.38.self_attn.v_proj.weight": "model-00017-of-00029.safetensors",
+    "layers.39.input_layernorm.weight": "model-00018-of-00029.safetensors",
+    "layers.39.mlp.down_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.39.mlp.gate_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.39.mlp.up_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.39.post_attention_layernorm.weight": "model-00018-of-00029.safetensors",
+    "layers.39.self_attn.k_proj.bias": "model-00018-of-00029.safetensors",
+    "layers.39.self_attn.k_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.39.self_attn.o_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.39.self_attn.q_proj.bias": "model-00018-of-00029.safetensors",
+    "layers.39.self_attn.q_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.39.self_attn.v_proj.bias": "model-00018-of-00029.safetensors",
+    "layers.39.self_attn.v_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.4.input_layernorm.weight": "model-00003-of-00029.safetensors",
+    "layers.4.mlp.down_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.4.mlp.gate_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.4.mlp.up_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.4.post_attention_layernorm.weight": "model-00003-of-00029.safetensors",
+    "layers.4.self_attn.k_proj.bias": "model-00003-of-00029.safetensors",
+    "layers.4.self_attn.k_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.4.self_attn.o_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.4.self_attn.q_proj.bias": "model-00003-of-00029.safetensors",
+    "layers.4.self_attn.q_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.4.self_attn.v_proj.bias": "model-00003-of-00029.safetensors",
+    "layers.4.self_attn.v_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.40.input_layernorm.weight": "model-00019-of-00029.safetensors",
+    "layers.40.mlp.down_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.40.mlp.gate_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.40.mlp.up_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.40.post_attention_layernorm.weight": "model-00019-of-00029.safetensors",
+    "layers.40.self_attn.k_proj.bias": "model-00018-of-00029.safetensors",
+    "layers.40.self_attn.k_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.40.self_attn.o_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.40.self_attn.q_proj.bias": "model-00018-of-00029.safetensors",
+    "layers.40.self_attn.q_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.40.self_attn.v_proj.bias": "model-00018-of-00029.safetensors",
+    "layers.40.self_attn.v_proj.weight": "model-00018-of-00029.safetensors",
+    "layers.41.input_layernorm.weight": "model-00019-of-00029.safetensors",
+    "layers.41.mlp.down_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.41.mlp.gate_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.41.mlp.up_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.41.post_attention_layernorm.weight": "model-00019-of-00029.safetensors",
+    "layers.41.self_attn.k_proj.bias": "model-00019-of-00029.safetensors",
+    "layers.41.self_attn.k_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.41.self_attn.o_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.41.self_attn.q_proj.bias": "model-00019-of-00029.safetensors",
+    "layers.41.self_attn.q_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.41.self_attn.v_proj.bias": "model-00019-of-00029.safetensors",
+    "layers.41.self_attn.v_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.42.input_layernorm.weight": "model-00020-of-00029.safetensors",
+    "layers.42.mlp.down_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.42.mlp.gate_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.42.mlp.up_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.42.post_attention_layernorm.weight": "model-00020-of-00029.safetensors",
+    "layers.42.self_attn.k_proj.bias": "model-00019-of-00029.safetensors",
+    "layers.42.self_attn.k_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.42.self_attn.o_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.42.self_attn.q_proj.bias": "model-00019-of-00029.safetensors",
+    "layers.42.self_attn.q_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.42.self_attn.v_proj.bias": "model-00019-of-00029.safetensors",
+    "layers.42.self_attn.v_proj.weight": "model-00019-of-00029.safetensors",
+    "layers.43.input_layernorm.weight": "model-00020-of-00029.safetensors",
+    "layers.43.mlp.down_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.43.mlp.gate_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.43.mlp.up_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.43.post_attention_layernorm.weight": "model-00020-of-00029.safetensors",
+    "layers.43.self_attn.k_proj.bias": "model-00020-of-00029.safetensors",
+    "layers.43.self_attn.k_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.43.self_attn.o_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.43.self_attn.q_proj.bias": "model-00020-of-00029.safetensors",
+    "layers.43.self_attn.q_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.43.self_attn.v_proj.bias": "model-00020-of-00029.safetensors",
+    "layers.43.self_attn.v_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.44.input_layernorm.weight": "model-00020-of-00029.safetensors",
+    "layers.44.mlp.down_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.44.mlp.gate_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.44.mlp.up_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.44.post_attention_layernorm.weight": "model-00020-of-00029.safetensors",
+    "layers.44.self_attn.k_proj.bias": "model-00020-of-00029.safetensors",
+    "layers.44.self_attn.k_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.44.self_attn.o_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.44.self_attn.q_proj.bias": "model-00020-of-00029.safetensors",
+    "layers.44.self_attn.q_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.44.self_attn.v_proj.bias": "model-00020-of-00029.safetensors",
+    "layers.44.self_attn.v_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.45.input_layernorm.weight": "model-00021-of-00029.safetensors",
+    "layers.45.mlp.down_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.45.mlp.gate_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.45.mlp.up_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.45.post_attention_layernorm.weight": "model-00021-of-00029.safetensors",
+    "layers.45.self_attn.k_proj.bias": "model-00020-of-00029.safetensors",
+    "layers.45.self_attn.k_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.45.self_attn.o_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.45.self_attn.q_proj.bias": "model-00020-of-00029.safetensors",
+    "layers.45.self_attn.q_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.45.self_attn.v_proj.bias": "model-00020-of-00029.safetensors",
+    "layers.45.self_attn.v_proj.weight": "model-00020-of-00029.safetensors",
+    "layers.46.input_layernorm.weight": "model-00021-of-00029.safetensors",
+    "layers.46.mlp.down_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.46.mlp.gate_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.46.mlp.up_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.46.post_attention_layernorm.weight": "model-00021-of-00029.safetensors",
+    "layers.46.self_attn.k_proj.bias": "model-00021-of-00029.safetensors",
+    "layers.46.self_attn.k_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.46.self_attn.o_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.46.self_attn.q_proj.bias": "model-00021-of-00029.safetensors",
+    "layers.46.self_attn.q_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.46.self_attn.v_proj.bias": "model-00021-of-00029.safetensors",
+    "layers.46.self_attn.v_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.47.input_layernorm.weight": "model-00022-of-00029.safetensors",
+    "layers.47.mlp.down_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.47.mlp.gate_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.47.mlp.up_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.47.post_attention_layernorm.weight": "model-00022-of-00029.safetensors",
+    "layers.47.self_attn.k_proj.bias": "model-00021-of-00029.safetensors",
+    "layers.47.self_attn.k_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.47.self_attn.o_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.47.self_attn.q_proj.bias": "model-00021-of-00029.safetensors",
+    "layers.47.self_attn.q_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.47.self_attn.v_proj.bias": "model-00021-of-00029.safetensors",
+    "layers.47.self_attn.v_proj.weight": "model-00021-of-00029.safetensors",
+    "layers.48.input_layernorm.weight": "model-00022-of-00029.safetensors",
+    "layers.48.mlp.down_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.48.mlp.gate_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.48.mlp.up_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.48.post_attention_layernorm.weight": "model-00022-of-00029.safetensors",
+    "layers.48.self_attn.k_proj.bias": "model-00022-of-00029.safetensors",
+    "layers.48.self_attn.k_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.48.self_attn.o_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.48.self_attn.q_proj.bias": "model-00022-of-00029.safetensors",
+    "layers.48.self_attn.q_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.48.self_attn.v_proj.bias": "model-00022-of-00029.safetensors",
+    "layers.48.self_attn.v_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.49.input_layernorm.weight": "model-00023-of-00029.safetensors",
+    "layers.49.mlp.down_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.49.mlp.gate_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.49.mlp.up_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.49.post_attention_layernorm.weight": "model-00023-of-00029.safetensors",
+    "layers.49.self_attn.k_proj.bias": "model-00022-of-00029.safetensors",
+    "layers.49.self_attn.k_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.49.self_attn.o_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.49.self_attn.q_proj.bias": "model-00022-of-00029.safetensors",
+    "layers.49.self_attn.q_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.49.self_attn.v_proj.bias": "model-00022-of-00029.safetensors",
+    "layers.49.self_attn.v_proj.weight": "model-00022-of-00029.safetensors",
+    "layers.5.input_layernorm.weight": "model-00004-of-00029.safetensors",
+    "layers.5.mlp.down_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.5.mlp.gate_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.5.mlp.up_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.5.post_attention_layernorm.weight": "model-00004-of-00029.safetensors",
+    "layers.5.self_attn.k_proj.bias": "model-00003-of-00029.safetensors",
+    "layers.5.self_attn.k_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.5.self_attn.o_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.5.self_attn.q_proj.bias": "model-00003-of-00029.safetensors",
+    "layers.5.self_attn.q_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.5.self_attn.v_proj.bias": "model-00003-of-00029.safetensors",
+    "layers.5.self_attn.v_proj.weight": "model-00003-of-00029.safetensors",
+    "layers.50.input_layernorm.weight": "model-00023-of-00029.safetensors",
+    "layers.50.mlp.down_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.50.mlp.gate_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.50.mlp.up_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.50.post_attention_layernorm.weight": "model-00023-of-00029.safetensors",
+    "layers.50.self_attn.k_proj.bias": "model-00023-of-00029.safetensors",
+    "layers.50.self_attn.k_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.50.self_attn.o_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.50.self_attn.q_proj.bias": "model-00023-of-00029.safetensors",
+    "layers.50.self_attn.q_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.50.self_attn.v_proj.bias": "model-00023-of-00029.safetensors",
+    "layers.50.self_attn.v_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.51.input_layernorm.weight": "model-00023-of-00029.safetensors",
+    "layers.51.mlp.down_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.51.mlp.gate_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.51.mlp.up_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.51.post_attention_layernorm.weight": "model-00023-of-00029.safetensors",
+    "layers.51.self_attn.k_proj.bias": "model-00023-of-00029.safetensors",
+    "layers.51.self_attn.k_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.51.self_attn.o_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.51.self_attn.q_proj.bias": "model-00023-of-00029.safetensors",
+    "layers.51.self_attn.q_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.51.self_attn.v_proj.bias": "model-00023-of-00029.safetensors",
+    "layers.51.self_attn.v_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.52.input_layernorm.weight": "model-00024-of-00029.safetensors",
+    "layers.52.mlp.down_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.52.mlp.gate_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.52.mlp.up_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.52.post_attention_layernorm.weight": "model-00024-of-00029.safetensors",
+    "layers.52.self_attn.k_proj.bias": "model-00023-of-00029.safetensors",
+    "layers.52.self_attn.k_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.52.self_attn.o_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.52.self_attn.q_proj.bias": "model-00023-of-00029.safetensors",
+    "layers.52.self_attn.q_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.52.self_attn.v_proj.bias": "model-00023-of-00029.safetensors",
+    "layers.52.self_attn.v_proj.weight": "model-00023-of-00029.safetensors",
+    "layers.53.input_layernorm.weight": "model-00024-of-00029.safetensors",
+    "layers.53.mlp.down_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.53.mlp.gate_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.53.mlp.up_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.53.post_attention_layernorm.weight": "model-00024-of-00029.safetensors",
+    "layers.53.self_attn.k_proj.bias": "model-00024-of-00029.safetensors",
+    "layers.53.self_attn.k_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.53.self_attn.o_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.53.self_attn.q_proj.bias": "model-00024-of-00029.safetensors",
+    "layers.53.self_attn.q_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.53.self_attn.v_proj.bias": "model-00024-of-00029.safetensors",
+    "layers.53.self_attn.v_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.54.input_layernorm.weight": "model-00025-of-00029.safetensors",
+    "layers.54.mlp.down_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.54.mlp.gate_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.54.mlp.up_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.54.post_attention_layernorm.weight": "model-00025-of-00029.safetensors",
+    "layers.54.self_attn.k_proj.bias": "model-00024-of-00029.safetensors",
+    "layers.54.self_attn.k_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.54.self_attn.o_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.54.self_attn.q_proj.bias": "model-00024-of-00029.safetensors",
+    "layers.54.self_attn.q_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.54.self_attn.v_proj.bias": "model-00024-of-00029.safetensors",
+    "layers.54.self_attn.v_proj.weight": "model-00024-of-00029.safetensors",
+    "layers.55.input_layernorm.weight": "model-00025-of-00029.safetensors",
+    "layers.55.mlp.down_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.55.mlp.gate_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.55.mlp.up_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.55.post_attention_layernorm.weight": "model-00025-of-00029.safetensors",
+    "layers.55.self_attn.k_proj.bias": "model-00025-of-00029.safetensors",
+    "layers.55.self_attn.k_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.55.self_attn.o_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.55.self_attn.q_proj.bias": "model-00025-of-00029.safetensors",
+    "layers.55.self_attn.q_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.55.self_attn.v_proj.bias": "model-00025-of-00029.safetensors",
+    "layers.55.self_attn.v_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.56.input_layernorm.weight": "model-00026-of-00029.safetensors",
+    "layers.56.mlp.down_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.56.mlp.gate_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.56.mlp.up_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.56.post_attention_layernorm.weight": "model-00026-of-00029.safetensors",
+    "layers.56.self_attn.k_proj.bias": "model-00025-of-00029.safetensors",
+    "layers.56.self_attn.k_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.56.self_attn.o_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.56.self_attn.q_proj.bias": "model-00025-of-00029.safetensors",
+    "layers.56.self_attn.q_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.56.self_attn.v_proj.bias": "model-00025-of-00029.safetensors",
+    "layers.56.self_attn.v_proj.weight": "model-00025-of-00029.safetensors",
+    "layers.57.input_layernorm.weight": "model-00026-of-00029.safetensors",
+    "layers.57.mlp.down_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.57.mlp.gate_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.57.mlp.up_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.57.post_attention_layernorm.weight": "model-00026-of-00029.safetensors",
+    "layers.57.self_attn.k_proj.bias": "model-00026-of-00029.safetensors",
+    "layers.57.self_attn.k_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.57.self_attn.o_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.57.self_attn.q_proj.bias": "model-00026-of-00029.safetensors",
+    "layers.57.self_attn.q_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.57.self_attn.v_proj.bias": "model-00026-of-00029.safetensors",
+    "layers.57.self_attn.v_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.58.input_layernorm.weight": "model-00026-of-00029.safetensors",
+    "layers.58.mlp.down_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.58.mlp.gate_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.58.mlp.up_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.58.post_attention_layernorm.weight": "model-00026-of-00029.safetensors",
+    "layers.58.self_attn.k_proj.bias": "model-00026-of-00029.safetensors",
+    "layers.58.self_attn.k_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.58.self_attn.o_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.58.self_attn.q_proj.bias": "model-00026-of-00029.safetensors",
+    "layers.58.self_attn.q_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.58.self_attn.v_proj.bias": "model-00026-of-00029.safetensors",
+    "layers.58.self_attn.v_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.59.input_layernorm.weight": "model-00027-of-00029.safetensors",
+    "layers.59.mlp.down_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.59.mlp.gate_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.59.mlp.up_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.59.post_attention_layernorm.weight": "model-00027-of-00029.safetensors",
+    "layers.59.self_attn.k_proj.bias": "model-00026-of-00029.safetensors",
+    "layers.59.self_attn.k_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.59.self_attn.o_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.59.self_attn.q_proj.bias": "model-00026-of-00029.safetensors",
+    "layers.59.self_attn.q_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.59.self_attn.v_proj.bias": "model-00026-of-00029.safetensors",
+    "layers.59.self_attn.v_proj.weight": "model-00026-of-00029.safetensors",
+    "layers.6.input_layernorm.weight": "model-00004-of-00029.safetensors",
+    "layers.6.mlp.down_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.6.mlp.gate_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.6.mlp.up_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.6.post_attention_layernorm.weight": "model-00004-of-00029.safetensors",
+    "layers.6.self_attn.k_proj.bias": "model-00004-of-00029.safetensors",
+    "layers.6.self_attn.k_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.6.self_attn.o_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.6.self_attn.q_proj.bias": "model-00004-of-00029.safetensors",
+    "layers.6.self_attn.q_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.6.self_attn.v_proj.bias": "model-00004-of-00029.safetensors",
+    "layers.6.self_attn.v_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.60.input_layernorm.weight": "model-00027-of-00029.safetensors",
+    "layers.60.mlp.down_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.60.mlp.gate_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.60.mlp.up_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.60.post_attention_layernorm.weight": "model-00027-of-00029.safetensors",
+    "layers.60.self_attn.k_proj.bias": "model-00027-of-00029.safetensors",
+    "layers.60.self_attn.k_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.60.self_attn.o_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.60.self_attn.q_proj.bias": "model-00027-of-00029.safetensors",
+    "layers.60.self_attn.q_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.60.self_attn.v_proj.bias": "model-00027-of-00029.safetensors",
+    "layers.60.self_attn.v_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.61.input_layernorm.weight": "model-00028-of-00029.safetensors",
+    "layers.61.mlp.down_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.61.mlp.gate_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.61.mlp.up_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.61.post_attention_layernorm.weight": "model-00028-of-00029.safetensors",
+    "layers.61.self_attn.k_proj.bias": "model-00027-of-00029.safetensors",
+    "layers.61.self_attn.k_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.61.self_attn.o_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.61.self_attn.q_proj.bias": "model-00027-of-00029.safetensors",
+    "layers.61.self_attn.q_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.61.self_attn.v_proj.bias": "model-00027-of-00029.safetensors",
+    "layers.61.self_attn.v_proj.weight": "model-00027-of-00029.safetensors",
+    "layers.62.input_layernorm.weight": "model-00028-of-00029.safetensors",
+    "layers.62.mlp.down_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.62.mlp.gate_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.62.mlp.up_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.62.post_attention_layernorm.weight": "model-00028-of-00029.safetensors",
+    "layers.62.self_attn.k_proj.bias": "model-00028-of-00029.safetensors",
+    "layers.62.self_attn.k_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.62.self_attn.o_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.62.self_attn.q_proj.bias": "model-00028-of-00029.safetensors",
+    "layers.62.self_attn.q_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.62.self_attn.v_proj.bias": "model-00028-of-00029.safetensors",
+    "layers.62.self_attn.v_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.63.input_layernorm.weight": "model-00029-of-00029.safetensors",
+    "layers.63.mlp.down_proj.weight": "model-00029-of-00029.safetensors",
+    "layers.63.mlp.gate_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.63.mlp.up_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.63.post_attention_layernorm.weight": "model-00029-of-00029.safetensors",
+    "layers.63.self_attn.k_proj.bias": "model-00028-of-00029.safetensors",
+    "layers.63.self_attn.k_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.63.self_attn.o_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.63.self_attn.q_proj.bias": "model-00028-of-00029.safetensors",
+    "layers.63.self_attn.q_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.63.self_attn.v_proj.bias": "model-00028-of-00029.safetensors",
+    "layers.63.self_attn.v_proj.weight": "model-00028-of-00029.safetensors",
+    "layers.7.input_layernorm.weight": "model-00005-of-00029.safetensors",
+    "layers.7.mlp.down_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.7.mlp.gate_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.7.mlp.up_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.7.post_attention_layernorm.weight": "model-00005-of-00029.safetensors",
+    "layers.7.self_attn.k_proj.bias": "model-00004-of-00029.safetensors",
+    "layers.7.self_attn.k_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.7.self_attn.o_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.7.self_attn.q_proj.bias": "model-00004-of-00029.safetensors",
+    "layers.7.self_attn.q_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.7.self_attn.v_proj.bias": "model-00004-of-00029.safetensors",
+    "layers.7.self_attn.v_proj.weight": "model-00004-of-00029.safetensors",
+    "layers.8.input_layernorm.weight": "model-00005-of-00029.safetensors",
+    "layers.8.mlp.down_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.8.mlp.gate_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.8.mlp.up_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.8.post_attention_layernorm.weight": "model-00005-of-00029.safetensors",
+    "layers.8.self_attn.k_proj.bias": "model-00005-of-00029.safetensors",
+    "layers.8.self_attn.k_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.8.self_attn.o_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.8.self_attn.q_proj.bias": "model-00005-of-00029.safetensors",
+    "layers.8.self_attn.q_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.8.self_attn.v_proj.bias": "model-00005-of-00029.safetensors",
+    "layers.8.self_attn.v_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.9.input_layernorm.weight": "model-00005-of-00029.safetensors",
+    "layers.9.mlp.down_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.9.mlp.gate_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.9.mlp.up_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.9.post_attention_layernorm.weight": "model-00005-of-00029.safetensors",
+    "layers.9.self_attn.k_proj.bias": "model-00005-of-00029.safetensors",
+    "layers.9.self_attn.k_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.9.self_attn.o_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.9.self_attn.q_proj.bias": "model-00005-of-00029.safetensors",
+    "layers.9.self_attn.q_proj.weight": "model-00005-of-00029.safetensors",
+    "layers.9.self_attn.v_proj.bias": "model-00005-of-00029.safetensors",
+    "layers.9.self_attn.v_proj.weight": "model-00005-of-00029.safetensors",
+    "norm.weight": "model-00029-of-00029.safetensors"
+  }
+}