Upload folder using huggingface_hub

README.md

@@ -0,0 +1,197 @@
+---
+base_model:
+- microsoft/phi-4
+license: mit
+license_link: https://huggingface.co/microsoft/phi-4/resolve/main/LICENSE
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- bnb-my-repo
+- phi
+- nlp
+- math
+- code
+- chat
+- conversational
+inference:
+  parameters:
+    temperature: 0
+widget:
+- messages:
+  - role: user
+    content: How should I explain the Internet?
+library_name: transformers
+---
+# microsoft/phi-4 (Quantized)
+
+## Description
+This model is a quantized version of the original model [`microsoft/phi-4`](https://huggingface.co/microsoft/phi-4). 
+
+It's quantized using the BitsAndBytes library to 4-bit using the [bnb-my-repo](https://huggingface.co/spaces/bnb-community/bnb-my-repo) space.
+
+## Quantization Details
+- **Quantization Type**: int4
+- **bnb_4bit_quant_type**: nf4
+- **bnb_4bit_use_double_quant**: True
+- **bnb_4bit_compute_dtype**: bfloat16
+- **bnb_4bit_quant_storage**: uint8
+
+
+
+# 📄 Original Model Information
+
+
+
+# Phi-4 Model Card 
+
+[Phi-4 Technical Report](https://arxiv.org/pdf/2412.08905)
+
+## Model Summary 
+
+|                         |                                                                               |     
+|-------------------------|-------------------------------------------------------------------------------|
+| **Developers**          | Microsoft Research                                                            |
+| **Description**         | `phi-4` is a state-of-the-art open model built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach was to ensure that small capable models were trained with data focused on high quality and advanced reasoning.<br><br>`phi-4` underwent a rigorous enhancement and alignment process, incorporating both supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures                |
+| **Architecture**        | 14B parameters, dense decoder-only Transformer model                          |
+| **Inputs**              | Text, best suited for prompts in the chat format                              |
+| **Context length**      | 16K tokens                                                                    |
+| **GPUs**                | 1920 H100-80G                                                                 |
+| **Training time**       | 21 days                                                                       |
+| **Training data**       | 9.8T tokens                                                                   |
+| **Outputs**             | Generated text in response to input                                           |
+| **Dates**               | October 2024 – November 2024                                                  |
+| **Status**              | Static model trained on an offline dataset with cutoff dates of June 2024 and earlier for publicly available data                                                                               |
+| **Release date**        | December 12, 2024                                                             |
+| **License**             | MIT                                                                         |
+
+## Intended Use 
+
+|                               |                                                                         |
+|-------------------------------|-------------------------------------------------------------------------|
+| **Primary Use Cases**         | Our model is designed to accelerate research on language models, for use as a building block for generative AI powered features. It provides uses for general purpose AI systems and applications (primarily in English) which require:<br><br>1. Memory/compute constrained environments.<br>2. Latency bound scenarios.<br>3. Reasoning and logic.                                                                       |
+| **Out-of-Scope Use Cases**    | Our models is not specifically designed or evaluated for all downstream purposes, thus:<br><br>1. Developers should consider common limitations of language models as they select use cases, and evaluate and mitigate for accuracy, safety, and fairness before using within a specific downstream use case, particularly for high-risk scenarios.<br>2. Developers should be aware of and adhere to applicable laws or regulations (including privacy, trade compliance laws, etc.) that are relevant to their use case, including the model’s focus on English.<br>3. Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the license the model is released under.                                                              |
+
+## Data Overview 
+
+### Training Datasets 
+
+Our training data is an extension of the data used for Phi-3 and includes a wide variety of sources from:
+
+1. Publicly available documents filtered rigorously for quality, selected high-quality educational data, and code.
+
+2. Newly created synthetic, “textbook-like” data for the purpose of teaching math, coding, common sense reasoning, general knowledge of the world (science, daily activities, theory of mind, etc.).
+
+3. Acquired academic books and Q&A datasets.
+
+4. High quality chat format supervised data covering various topics to reflect human preferences on different aspects such as instruct-following, truthfulness, honesty and helpfulness.
+
+Multilingual data constitutes about 8% of our overall data. We are focusing on the quality of data that could potentially improve the reasoning ability for the model, and we filter the publicly available documents to contain the correct level of knowledge.
+
+#### Benchmark datasets 
+
+We evaluated `phi-4` using [OpenAI’s SimpleEval](https://github.com/openai/simple-evals) and our own internal benchmarks to understand the model’s capabilities, more specifically: 
+
+* **MMLU:** Popular aggregated dataset for multitask language understanding.
+
+* **MATH:** Challenging competition math problems.
+
+* **GPQA:** Complex, graduate-level science questions.
+
+* **DROP:** Complex comprehension and reasoning.
+
+* **MGSM:** Multi-lingual grade-school math.
+
+* **HumanEval:** Functional code generation.
+
+* **SimpleQA:** Factual responses.
+
+## Safety 
+
+### Approach 
+
+`phi-4` has adopted a robust safety post-training approach. This approach leverages a variety of both open-source and in-house generated synthetic datasets. The overall technique employed to do the safety alignment is a combination of SFT (Supervised Fine-Tuning) and iterative DPO (Direct Preference Optimization), including publicly available datasets focusing on helpfulness and harmlessness as well as various questions and answers targeted to multiple safety categories. 
+
+### Safety Evaluation and Red-Teaming 
+
+Prior to release, `phi-4` followed a multi-faceted evaluation approach. Quantitative evaluation was conducted with multiple open-source safety benchmarks and in-house tools utilizing adversarial conversation simulation. For qualitative safety evaluation, we collaborated with the independent AI Red Team (AIRT) at Microsoft to assess safety risks posed by `phi-4` in both average and adversarial user scenarios. In the average user scenario, AIRT emulated typical single-turn and multi-turn interactions to identify potentially risky behaviors. The adversarial user scenario tested a wide range of techniques aimed at intentionally subverting the model’s safety training including jailbreaks, encoding-based attacks, multi-turn attacks, and adversarial suffix attacks.   
+
+Please refer to the technical report for more details on safety alignment. 
+
+## Model Quality
+
+To understand the capabilities, we compare `phi-4` with a set of models over OpenAI’s SimpleEval benchmark. 
+
+At the high-level overview of the model quality on representative benchmarks. For the table below, higher numbers indicate better performance: 
+
+| **Category**                 | **Benchmark** | **phi-4** (14B) | **phi-3** (14B) | **Qwen 2.5** (14B instruct) | **GPT-4o-mini** | **Llama-3.3** (70B instruct) | **Qwen 2.5** (72B instruct) | **GPT-4o** |
+|------------------------------|---------------|-----------|-----------------|----------------------|----------------------|--------------------|-------------------|-----------------|
+| Popular Aggregated Benchmark | MMLU          | 84.8      | 77.9            | 79.9                 | 81.8                 | 86.3               | 85.3              | **88.1**            |
+| Science                      | GPQA          | **56.1**      | 31.2            | 42.9                 | 40.9                 | 49.1               | 49.0              | 50.6            |
+| Math                         | MGSM<br>MATH  | 80.6<br>**80.4** | 53.5<br>44.6 | 79.6<br>75.6 | 86.5<br>73.0 | 89.1<br>66.3* | 87.3<br>80.0              | **90.4**<br>74.6            |
+| Code Generation              | HumanEval     | 82.6      | 67.8            | 72.1                 | 86.2                 | 78.9*               | 80.4              | **90.6**            |
+| Factual Knowledge            | SimpleQA      | 3.0       | 7.6            | 5.4                 | 9.9                  | 20.9               | 10.2              | **39.4**             |
+| Reasoning                    | DROP          | 75.5      | 68.3            | 85.5                 | 79.3                 | **90.2**               | 76.7              | 80.9            |
+
+\* These scores are lower than those reported by Meta, perhaps because simple-evals has a strict formatting requirement that Llama models have particular trouble following. We use the simple-evals framework because it is reproducible, but Meta reports 77 for MATH and 88 for HumanEval on Llama-3.3-70B.
+
+## Usage
+
+### Input Formats
+
+Given the nature of the training data, `phi-4` is best suited for prompts using the chat format as follows: 
+
+```bash
+<|im_start|>system<|im_sep|>
+You are a medieval knight and must provide explanations to modern people.<|im_end|>
+<|im_start|>user<|im_sep|>
+How should I explain the Internet?<|im_end|>
+<|im_start|>assistant<|im_sep|>
+```
+
+### With `transformers`
+
+```python
+import transformers
+
+pipeline = transformers.pipeline(
+    "text-generation",
+    model="microsoft/phi-4",
+    model_kwargs={"torch_dtype": "auto"},
+    device_map="auto",
+)
+
+messages = [
+    {"role": "system", "content": "You are a medieval knight and must provide explanations to modern people."},
+    {"role": "user", "content": "How should I explain the Internet?"},
+]
+
+outputs = pipeline(messages, max_new_tokens=128)
+print(outputs[0]["generated_text"][-1])
+```
+
+## Responsible AI Considerations
+
+Like other language models, `phi-4` can potentially behave in ways that are unfair, unreliable, or offensive. Some of the limiting behaviors to be aware of include:   
+
+* **Quality of Service:** The model is trained primarily on English text. Languages other than English will experience worse performance. English language varieties with less representation in the training data might experience worse performance than standard American English. `phi-4` is not intended to support multilingual use. 
+
+* **Representation of Harms & Perpetuation of Stereotypes:** These models can over- or under-represent groups of people, erase representation of some groups, or reinforce demeaning or negative stereotypes. Despite safety post-training, these limitations may still be present due to differing levels of representation of different groups or prevalence of examples of negative stereotypes in training data that reflect real-world patterns and societal biases.  
+
+* **Inappropriate or Offensive Content:** These models may produce other types of inappropriate or offensive content, which may make it inappropriate to deploy for sensitive contexts without additional mitigations that are specific to the use case.  
+
+* **Information Reliability:** Language models can generate nonsensical content or fabricate content that might sound reasonable but is inaccurate or outdated.   
+
+* **Limited Scope for Code:** Majority of `phi-4` training data is based in Python and uses common packages such as `typing`, `math`, `random`, `collections`, `datetime`, `itertools`. If the model generates Python scripts that utilize other packages or scripts in other languages, we strongly recommend users manually verify all API uses.  
+
+Developers should apply responsible AI best practices and are responsible for ensuring that a specific use case complies with relevant laws and regulations (e.g. privacy, trade, etc.). Using safety services like [Azure AI Content Safety](https://azure.microsoft.com/en-us/products/ai-services/ai-content-safety) that have advanced guardrails is highly recommended. Important areas for consideration include:
+
+* **Allocation:** Models may not be suitable for scenarios that could have consequential impact on legal status or the allocation of resources or life opportunities (ex: housing, employment, credit, etc.) without further assessments and additional debiasing techniques. 
+
+* **High-Risk Scenarios:** Developers should assess suitability of using models in high-risk scenarios where unfair, unreliable or offensive outputs might be extremely costly or lead to harm. This includes providing advice in sensitive or expert domains where accuracy and reliability are critical (ex: legal or health advice). Additional safeguards should be implemented at the application level according to the deployment context.  
+
+* **Misinformation:** Models may produce inaccurate information. Developers should follow transparency best practices and inform end-users they are interacting with an AI system. At the application level, developers can build feedback mechanisms and pipelines to ground responses in use-case specific, contextual information, a technique known as Retrieval Augmented Generation (RAG).    
+
+* **Generation of Harmful Content:** Developers should assess outputs for their context and use available safety classifiers or custom solutions appropriate for their use case.  
+
+* **Misuse:** Other forms of misuse such as fraud, spam, or malware production may be possible, and developers should ensure that their applications do not violate applicable laws and regulations.

config.json

@@ -0,0 +1,48 @@
+{
+  "_name_or_path": "microsoft/phi-4",
+  "architectures": [
+    "Phi3Model"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 100257,
+  "embd_pdrop": 0.0,
+  "eos_token_id": 100265,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 17920,
+  "max_position_embeddings": 16384,
+  "model_type": "phi3",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 10,
+  "original_max_position_embeddings": 16384,
+  "pad_token_id": 100349,
+  "partial_rotary_factor": 1.0,
+  "quantization_config": {
+    "_load_in_4bit": true,
+    "_load_in_8bit": false,
+    "bnb_4bit_compute_dtype": "bfloat16",
+    "bnb_4bit_quant_storage": "uint8",
+    "bnb_4bit_quant_type": "nf4",
+    "bnb_4bit_use_double_quant": true,
+    "llm_int8_enable_fp32_cpu_offload": false,
+    "llm_int8_has_fp16_weight": false,
+    "llm_int8_skip_modules": null,
+    "llm_int8_threshold": 6.0,
+    "load_in_4bit": true,
+    "load_in_8bit": false,
+    "quant_method": "bitsandbytes"
+  },
+  "resid_pdrop": 0.0,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 250000,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.49.0",
+  "use_cache": true,
+  "vocab_size": 100352
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a286cbc5bfe6452921aa401944213ffc44b1e74b5cedad94b3d757fd180eb874
+size 4929678823

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fa8e3b10e27d5a6b47cc74c2d6b9275dd94da6bcb5c390d07ea2d0f7fdad638
+size 3131140124

model.safetensors.index.json

@@ -0,0 +1,1049 @@
+{
+  "metadata": {
+    "total_size": 8060699627
+  },
+  "weight_map": {
+    "embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.0.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.0.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.1.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.1.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.10.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.10.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.11.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.11.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.12.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.12.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.13.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.13.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.14.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.14.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.15.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.15.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.16.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.16.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.17.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.17.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.18.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.18.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.19.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.19.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.2.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.2.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.20.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.20.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.21.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.21.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.21.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.22.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.22.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.22.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.23.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.23.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.24.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.24.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.25.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.25.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.26.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.26.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.27.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.27.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.28.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.28.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.29.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.29.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.3.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.3.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.30.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.30.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.31.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.31.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.32.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.32.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.33.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.33.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.34.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.34.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.35.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.35.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.36.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.36.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.36.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.36.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.37.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.37.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.37.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.37.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.38.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.38.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.38.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.38.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.39.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.down_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.down_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.down_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.down_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.gate_up_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.gate_up_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.gate_up_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.gate_up_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.gate_up_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.39.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.39.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.o_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.o_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.o_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.o_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.qkv_proj.weight": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.qkv_proj.weight.absmax": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.qkv_proj.weight.nested_absmax": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.qkv_proj.weight.nested_quant_map": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.qkv_proj.weight.quant_map": "model-00002-of-00002.safetensors",
+    "layers.39.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00002-of-00002.safetensors",
+    "layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.4.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.4.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.5.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.5.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.6.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.6.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.7.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.7.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.8.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.8.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.down_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.down_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.down_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.down_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.down_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.gate_up_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.gate_up_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.gate_up_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.gate_up_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.gate_up_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.9.mlp.gate_up_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.o_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.o_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.o_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.o_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.o_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.qkv_proj.weight": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.qkv_proj.weight.absmax": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.qkv_proj.weight.nested_absmax": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.qkv_proj.weight.nested_quant_map": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.qkv_proj.weight.quant_map": "model-00001-of-00002.safetensors",
+    "layers.9.self_attn.qkv_proj.weight.quant_state.bitsandbytes__nf4": "model-00001-of-00002.safetensors",
+    "norm.weight": "model-00002-of-00002.safetensors"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|dummy_85|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,782 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "100256": {
+      "content": "<|dummy_0|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100257": {
+      "content": "<|endoftext|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100258": {
+      "content": "<|fim_prefix|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100259": {
+      "content": "<|fim_middle|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100260": {
+      "content": "<|fim_suffix|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100261": {
+      "content": "<|dummy_1|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100262": {
+      "content": "<|dummy_2|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100263": {
+      "content": "<|dummy_3|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100264": {
+      "content": "<|im_start|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100265": {
+      "content": "<|im_end|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100266": {
+      "content": "<|im_sep|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100267": {
+      "content": "<|dummy_4|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100268": {
+      "content": "<|dummy_5|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100269": {
+      "content": "<|dummy_6|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100270": {
+      "content": "<|dummy_7|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100271": {
+      "content": "<|dummy_8|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100272": {
+      "content": "<|dummy_9|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100273": {
+      "content": "<|dummy_10|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100274": {
+      "content": "<|dummy_11|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100275": {
+      "content": "<|dummy_12|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100276": {
+      "content": "<|endofprompt|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100277": {
+      "content": "<|dummy_13|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100278": {
+      "content": "<|dummy_14|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100279": {
+      "content": "<|dummy_15|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100280": {
+      "content": "<|dummy_16|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100281": {
+      "content": "<|dummy_17|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100282": {
+      "content": "<|dummy_18|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100283": {
+      "content": "<|dummy_19|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100284": {
+      "content": "<|dummy_20|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100285": {
+      "content": "<|dummy_21|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100286": {
+      "content": "<|dummy_22|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100287": {
+      "content": "<|dummy_23|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100288": {
+      "content": "<|dummy_24|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100289": {
+      "content": "<|dummy_25|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100290": {
+      "content": "<|dummy_26|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100291": {
+      "content": "<|dummy_27|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100292": {
+      "content": "<|dummy_28|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100293": {
+      "content": "<|dummy_29|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100294": {
+      "content": "<|dummy_30|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100295": {
+      "content": "<|dummy_31|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100296": {
+      "content": "<|dummy_32|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100297": {
+      "content": "<|dummy_33|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100298": {
+      "content": "<|dummy_34|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100299": {
+      "content": "<|dummy_35|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100300": {
+      "content": "<|dummy_36|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100301": {
+      "content": "<|dummy_37|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100302": {
+      "content": "<|dummy_38|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100303": {
+      "content": "<|dummy_39|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100304": {
+      "content": "<|dummy_40|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100305": {
+      "content": "<|dummy_41|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100306": {
+      "content": "<|dummy_42|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100307": {
+      "content": "<|dummy_43|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100308": {
+      "content": "<|dummy_44|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100309": {
+      "content": "<|dummy_45|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100310": {
+      "content": "<|dummy_46|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100311": {
+      "content": "<|dummy_47|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100312": {
+      "content": "<|dummy_48|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100313": {
+      "content": "<|dummy_49|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100314": {
+      "content": "<|dummy_50|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100315": {
+      "content": "<|dummy_51|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100316": {
+      "content": "<|dummy_52|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100317": {
+      "content": "<|dummy_53|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100318": {
+      "content": "<|dummy_54|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100319": {
+      "content": "<|dummy_55|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100320": {
+      "content": "<|dummy_56|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100321": {
+      "content": "<|dummy_57|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100322": {
+      "content": "<|dummy_58|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100323": {
+      "content": "<|dummy_59|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100324": {
+      "content": "<|dummy_60|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100325": {
+      "content": "<|dummy_61|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100326": {
+      "content": "<|dummy_62|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100327": {
+      "content": "<|dummy_63|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100328": {
+      "content": "<|dummy_64|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100329": {
+      "content": "<|dummy_65|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100330": {
+      "content": "<|dummy_66|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100331": {
+      "content": "<|dummy_67|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100332": {
+      "content": "<|dummy_68|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100333": {
+      "content": "<|dummy_69|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100334": {
+      "content": "<|dummy_70|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100335": {
+      "content": "<|dummy_71|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100336": {
+      "content": "<|dummy_72|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100337": {
+      "content": "<|dummy_73|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100338": {
+      "content": "<|dummy_74|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100339": {
+      "content": "<|dummy_75|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100340": {
+      "content": "<|dummy_76|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100341": {
+      "content": "<|dummy_77|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100342": {
+      "content": "<|dummy_78|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100343": {
+      "content": "<|dummy_79|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100344": {
+      "content": "<|dummy_80|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100345": {
+      "content": "<|dummy_81|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100346": {
+      "content": "<|dummy_82|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100347": {
+      "content": "<|dummy_83|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100348": {
+      "content": "<|dummy_84|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100349": {
+      "content": "<|dummy_85|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100350": {
+      "content": "<|dummy_86|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100351": {
+      "content": "<|dummy_87|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "{% for message in messages %}{% if (message['role'] == 'system') %}{{'<|im_start|>system<|im_sep|>' + message['content'] + '<|im_end|>'}}{% elif (message['role'] == 'user') %}{{'<|im_start|>user<|im_sep|>' + message['content'] + '<|im_end|>'}}{% elif (message['role'] == 'assistant') %}{{'<|im_start|>assistant<|im_sep|>' + message['content'] + '<|im_end|>'}}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant<|im_sep|>' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "extra_special_tokens": {},
+  "model_max_length": 16384,
+  "pad_token": "<|dummy_85|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}