Initial model upload

README.md

@@ -0,0 +1,61 @@
+---
+base_model: alexsherstinsky/Mistral-7B-v0.1-sharded
+library_name: peft
+license: other
+tags:
+- llama-factory
+- lora
+- generated_from_trainer
+model-index:
+- name: 4k_train_2024-10-16-09-48-21
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# 4k_train_2024-10-16-09-48-21
+
+This model is a fine-tuned version of [alexsherstinsky/Mistral-7B-v0.1-sharded](https://huggingface.co/alexsherstinsky/Mistral-7B-v0.1-sharded) on the identity dataset.
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 2
+- eval_batch_size: 8
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 2
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 32
+- total_eval_batch_size: 16
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- num_epochs: 6.0
+
+### Training results
+
+
+
+### Framework versions
+
+- PEFT 0.12.0
+- Transformers 4.45.0
+- Pytorch 2.3.1+cu121
+- Datasets 2.21.0
+- Tokenizers 0.20.1

adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "alexsherstinsky/Mistral-7B-v0.1-sharded",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "q_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7364cf78be4e561405a20ffc94cc18c1a708a08b2b9a90290410d53241c5eda0
+size 83945296

all_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 5.971724787935909,
+    "num_input_tokens_seen": 9124912,
+    "total_flos": 3.904530641814815e+17,
+    "train_loss": 1.665087498799719,
+    "train_runtime": 5809.5562,
+    "train_samples_per_second": 4.383,
+    "train_steps_per_second": 0.136
+}

checkpoint-100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: alexsherstinsky/Mistral-7B-v0.1-sharded
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-100/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "alexsherstinsky/Mistral-7B-v0.1-sharded",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "q_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f23221dcb315c1c595944eb929760210575087150684bf53603be4664f30cc0e
+size 83945296

checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f28b7f8c5a1bd76e50c03ace7b424bb5682e6b7bec42f4486abb39d87f9e3b9e
+size 168149074

checkpoint-100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8d6a959372d5e0c2ea025dd26c9d0ad2046fce19352056cae8074dcbd0a6fd4
+size 14512

checkpoint-100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f68a37892a1b445d21bb35cc10bf7a058a6f9ec8c363f5ed156ff4f49d90fb6
+size 14512

checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:007585e9be6fcea10724fc5c4f995ce0d7c1a0cb64ea3e579daa75bb93d29802
+size 1064

checkpoint-100/special_tokens_map.json

@@ -0,0 +1,29 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-100/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-100/tokenizer_config.json

@@ -0,0 +1,50 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{{ '<s>' }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}{% if system_message is defined %}{{ system_message }}{% endif %}{% for message in loop_messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '[INST] ' + content + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ content + '</s>' }}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

checkpoint-100/trainer_state.json

@@ -0,0 +1,193 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.7540056550424128,
+  "eval_steps": 500,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03770028275212064,
+      "grad_norm": 2.8582637310028076,
+      "learning_rate": 4.9995083170283816e-05,
+      "loss": 2.995,
+      "num_input_tokens_seen": 57008,
+      "step": 5
+    },
+    {
+      "epoch": 0.07540056550424128,
+      "grad_norm": 2.4749696254730225,
+      "learning_rate": 4.998033461515242e-05,
+      "loss": 2.7127,
+      "num_input_tokens_seen": 117936,
+      "step": 10
+    },
+    {
+      "epoch": 0.11310084825636192,
+      "grad_norm": 2.3738160133361816,
+      "learning_rate": 4.9955760135896534e-05,
+      "loss": 2.6302,
+      "num_input_tokens_seen": 176288,
+      "step": 15
+    },
+    {
+      "epoch": 0.15080113100848255,
+      "grad_norm": 2.2132370471954346,
+      "learning_rate": 4.992136939879856e-05,
+      "loss": 2.5818,
+      "num_input_tokens_seen": 229712,
+      "step": 20
+    },
+    {
+      "epoch": 0.1885014137606032,
+      "grad_norm": 2.444084882736206,
+      "learning_rate": 4.9877175931330346e-05,
+      "loss": 2.4913,
+      "num_input_tokens_seen": 288784,
+      "step": 25
+    },
+    {
+      "epoch": 0.22620169651272384,
+      "grad_norm": 2.190948009490967,
+      "learning_rate": 4.982319711683221e-05,
+      "loss": 2.4694,
+      "num_input_tokens_seen": 347712,
+      "step": 30
+    },
+    {
+      "epoch": 0.2639019792648445,
+      "grad_norm": 1.9827752113342285,
+      "learning_rate": 4.975945418767529e-05,
+      "loss": 2.4525,
+      "num_input_tokens_seen": 404272,
+      "step": 35
+    },
+    {
+      "epoch": 0.3016022620169651,
+      "grad_norm": 2.3744397163391113,
+      "learning_rate": 4.968597221690986e-05,
+      "loss": 2.4745,
+      "num_input_tokens_seen": 461632,
+      "step": 40
+    },
+    {
+      "epoch": 0.3393025447690858,
+      "grad_norm": 2.5817763805389404,
+      "learning_rate": 4.96027801084029e-05,
+      "loss": 2.4637,
+      "num_input_tokens_seen": 518016,
+      "step": 45
+    },
+    {
+      "epoch": 0.3770028275212064,
+      "grad_norm": 2.2218539714813232,
+      "learning_rate": 4.950991058546893e-05,
+      "loss": 2.5011,
+      "num_input_tokens_seen": 579312,
+      "step": 50
+    },
+    {
+      "epoch": 0.41470311027332707,
+      "grad_norm": 2.5139260292053223,
+      "learning_rate": 4.940740017799833e-05,
+      "loss": 2.4397,
+      "num_input_tokens_seen": 636448,
+      "step": 55
+    },
+    {
+      "epoch": 0.4524033930254477,
+      "grad_norm": 2.589820623397827,
+      "learning_rate": 4.929528920808854e-05,
+      "loss": 2.4109,
+      "num_input_tokens_seen": 693504,
+      "step": 60
+    },
+    {
+      "epoch": 0.49010367577756836,
+      "grad_norm": 2.8880276679992676,
+      "learning_rate": 4.917362177418342e-05,
+      "loss": 2.4228,
+      "num_input_tokens_seen": 752160,
+      "step": 65
+    },
+    {
+      "epoch": 0.527803958529689,
+      "grad_norm": 2.480609178543091,
+      "learning_rate": 4.904244573372733e-05,
+      "loss": 2.4391,
+      "num_input_tokens_seen": 811296,
+      "step": 70
+    },
+    {
+      "epoch": 0.5655042412818096,
+      "grad_norm": 2.6764094829559326,
+      "learning_rate": 4.8901812684340564e-05,
+      "loss": 2.4552,
+      "num_input_tokens_seen": 867696,
+      "step": 75
+    },
+    {
+      "epoch": 0.6032045240339302,
+      "grad_norm": 2.666821002960205,
+      "learning_rate": 4.8751777943523634e-05,
+      "loss": 2.3658,
+      "num_input_tokens_seen": 924944,
+      "step": 80
+    },
+    {
+      "epoch": 0.6409048067860509,
+      "grad_norm": 2.884999990463257,
+      "learning_rate": 4.8592400526898314e-05,
+      "loss": 2.359,
+      "num_input_tokens_seen": 978768,
+      "step": 85
+    },
+    {
+      "epoch": 0.6786050895381716,
+      "grad_norm": 2.58659291267395,
+      "learning_rate": 4.842374312499405e-05,
+      "loss": 2.4228,
+      "num_input_tokens_seen": 1034544,
+      "step": 90
+    },
+    {
+      "epoch": 0.7163053722902922,
+      "grad_norm": 2.6272594928741455,
+      "learning_rate": 4.824587207858888e-05,
+      "loss": 2.32,
+      "num_input_tokens_seen": 1091776,
+      "step": 95
+    },
+    {
+      "epoch": 0.7540056550424128,
+      "grad_norm": 2.6761183738708496,
+      "learning_rate": 4.805885735261454e-05,
+      "loss": 2.3393,
+      "num_input_tokens_seen": 1153056,
+      "step": 100
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 792,
+  "num_input_tokens_seen": 1153056,
+  "num_train_epochs": 6,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.933902388114227e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71c7be1d061ec911f0c413e914d833bd67f4314dfb89be07d4786b40787b7e69
+size 5432

checkpoint-200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: alexsherstinsky/Mistral-7B-v0.1-sharded
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-200/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "alexsherstinsky/Mistral-7B-v0.1-sharded",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "q_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66bad24afee2d3a2dc7119dd67075dfff53ccce58bd5b2a8c6b0cf585f82b52e
+size 83945296

checkpoint-200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf7954e8c57aebfffc1a42fe87ea8345bb888ab6c9cd590bd6d16211606f654f
+size 168149074

checkpoint-200/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cb795a5cea0baa625c50007a6c9da09c6bbb5c16b560424070384a479e7d8a6
+size 14512

checkpoint-200/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f19604377bd828eb366c68946ad997a4ff4d69beaeea93ee58915135768ec63
+size 14512

checkpoint-200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa006138eee6fa0ff47a9911fb5414a64744afc4fa750d29d1a25fc9da929eba
+size 1064

checkpoint-200/special_tokens_map.json

@@ -0,0 +1,29 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-200/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-200/tokenizer_config.json

@@ -0,0 +1,50 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{{ '<s>' }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}{% if system_message is defined %}{{ system_message }}{% endif %}{% for message in loop_messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '[INST] ' + content + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ content + '</s>' }}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

checkpoint-200/trainer_state.json

@@ -0,0 +1,353 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.5080113100848256,
+  "eval_steps": 500,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03770028275212064,
+      "grad_norm": 2.8582637310028076,
+      "learning_rate": 4.9995083170283816e-05,
+      "loss": 2.995,
+      "num_input_tokens_seen": 57008,
+      "step": 5
+    },
+    {
+      "epoch": 0.07540056550424128,
+      "grad_norm": 2.4749696254730225,
+      "learning_rate": 4.998033461515242e-05,
+      "loss": 2.7127,
+      "num_input_tokens_seen": 117936,
+      "step": 10
+    },
+    {
+      "epoch": 0.11310084825636192,
+      "grad_norm": 2.3738160133361816,
+      "learning_rate": 4.9955760135896534e-05,
+      "loss": 2.6302,
+      "num_input_tokens_seen": 176288,
+      "step": 15
+    },
+    {
+      "epoch": 0.15080113100848255,
+      "grad_norm": 2.2132370471954346,
+      "learning_rate": 4.992136939879856e-05,
+      "loss": 2.5818,
+      "num_input_tokens_seen": 229712,
+      "step": 20
+    },
+    {
+      "epoch": 0.1885014137606032,
+      "grad_norm": 2.444084882736206,
+      "learning_rate": 4.9877175931330346e-05,
+      "loss": 2.4913,
+      "num_input_tokens_seen": 288784,
+      "step": 25
+    },
+    {
+      "epoch": 0.22620169651272384,
+      "grad_norm": 2.190948009490967,
+      "learning_rate": 4.982319711683221e-05,
+      "loss": 2.4694,
+      "num_input_tokens_seen": 347712,
+      "step": 30
+    },
+    {
+      "epoch": 0.2639019792648445,
+      "grad_norm": 1.9827752113342285,
+      "learning_rate": 4.975945418767529e-05,
+      "loss": 2.4525,
+      "num_input_tokens_seen": 404272,
+      "step": 35
+    },
+    {
+      "epoch": 0.3016022620169651,
+      "grad_norm": 2.3744397163391113,
+      "learning_rate": 4.968597221690986e-05,
+      "loss": 2.4745,
+      "num_input_tokens_seen": 461632,
+      "step": 40
+    },
+    {
+      "epoch": 0.3393025447690858,
+      "grad_norm": 2.5817763805389404,
+      "learning_rate": 4.96027801084029e-05,
+      "loss": 2.4637,
+      "num_input_tokens_seen": 518016,
+      "step": 45
+    },
+    {
+      "epoch": 0.3770028275212064,
+      "grad_norm": 2.2218539714813232,
+      "learning_rate": 4.950991058546893e-05,
+      "loss": 2.5011,
+      "num_input_tokens_seen": 579312,
+      "step": 50
+    },
+    {
+      "epoch": 0.41470311027332707,
+      "grad_norm": 2.5139260292053223,
+      "learning_rate": 4.940740017799833e-05,
+      "loss": 2.4397,
+      "num_input_tokens_seen": 636448,
+      "step": 55
+    },
+    {
+      "epoch": 0.4524033930254477,
+      "grad_norm": 2.589820623397827,
+      "learning_rate": 4.929528920808854e-05,
+      "loss": 2.4109,
+      "num_input_tokens_seen": 693504,
+      "step": 60
+    },
+    {
+      "epoch": 0.49010367577756836,
+      "grad_norm": 2.8880276679992676,
+      "learning_rate": 4.917362177418342e-05,
+      "loss": 2.4228,
+      "num_input_tokens_seen": 752160,
+      "step": 65
+    },
+    {
+      "epoch": 0.527803958529689,
+      "grad_norm": 2.480609178543091,
+      "learning_rate": 4.904244573372733e-05,
+      "loss": 2.4391,
+      "num_input_tokens_seen": 811296,
+      "step": 70
+    },
+    {
+      "epoch": 0.5655042412818096,
+      "grad_norm": 2.6764094829559326,
+      "learning_rate": 4.8901812684340564e-05,
+      "loss": 2.4552,
+      "num_input_tokens_seen": 867696,
+      "step": 75
+    },
+    {
+      "epoch": 0.6032045240339302,
+      "grad_norm": 2.666821002960205,
+      "learning_rate": 4.8751777943523634e-05,
+      "loss": 2.3658,
+      "num_input_tokens_seen": 924944,
+      "step": 80
+    },
+    {
+      "epoch": 0.6409048067860509,
+      "grad_norm": 2.884999990463257,
+      "learning_rate": 4.8592400526898314e-05,
+      "loss": 2.359,
+      "num_input_tokens_seen": 978768,
+      "step": 85
+    },
+    {
+      "epoch": 0.6786050895381716,
+      "grad_norm": 2.58659291267395,
+      "learning_rate": 4.842374312499405e-05,
+      "loss": 2.4228,
+      "num_input_tokens_seen": 1034544,
+      "step": 90
+    },
+    {
+      "epoch": 0.7163053722902922,
+      "grad_norm": 2.6272594928741455,
+      "learning_rate": 4.824587207858888e-05,
+      "loss": 2.32,
+      "num_input_tokens_seen": 1091776,
+      "step": 95
+    },
+    {
+      "epoch": 0.7540056550424128,
+      "grad_norm": 2.6761183738708496,
+      "learning_rate": 4.805885735261454e-05,
+      "loss": 2.3393,
+      "num_input_tokens_seen": 1153056,
+      "step": 100
+    },
+    {
+      "epoch": 0.7917059377945335,
+      "grad_norm": 2.721721887588501,
+      "learning_rate": 4.786277250863599e-05,
+      "loss": 2.2595,
+      "num_input_tokens_seen": 1211808,
+      "step": 105
+    },
+    {
+      "epoch": 0.8294062205466541,
+      "grad_norm": 2.718644618988037,
+      "learning_rate": 4.765769467591625e-05,
+      "loss": 2.3171,
+      "num_input_tokens_seen": 1273600,
+      "step": 110
+    },
+    {
+      "epoch": 0.8671065032987747,
+      "grad_norm": 2.6614394187927246,
+      "learning_rate": 4.744370452107789e-05,
+      "loss": 2.2949,
+      "num_input_tokens_seen": 1331392,
+      "step": 115
+    },
+    {
+      "epoch": 0.9048067860508954,
+      "grad_norm": 2.851165771484375,
+      "learning_rate": 4.722088621637309e-05,
+      "loss": 2.3818,
+      "num_input_tokens_seen": 1387664,
+      "step": 120
+    },
+    {
+      "epoch": 0.942507068803016,
+      "grad_norm": 2.692765235900879,
+      "learning_rate": 4.698932740657479e-05,
+      "loss": 2.2494,
+      "num_input_tokens_seen": 1445968,
+      "step": 125
+    },
+    {
+      "epoch": 0.9802073515551367,
+      "grad_norm": 2.7365992069244385,
+      "learning_rate": 4.6749119174501975e-05,
+      "loss": 2.3593,
+      "num_input_tokens_seen": 1497744,
+      "step": 130
+    },
+    {
+      "epoch": 1.0179076343072573,
+      "grad_norm": 2.6355655193328857,
+      "learning_rate": 4.6500356005192514e-05,
+      "loss": 2.1892,
+      "num_input_tokens_seen": 1549008,
+      "step": 135
+    },
+    {
+      "epoch": 1.055607917059378,
+      "grad_norm": 2.8112802505493164,
+      "learning_rate": 4.6243135748737864e-05,
+      "loss": 2.1753,
+      "num_input_tokens_seen": 1605152,
+      "step": 140
+    },
+    {
+      "epoch": 1.0933081998114986,
+      "grad_norm": 3.1792027950286865,
+      "learning_rate": 4.597755958179406e-05,
+      "loss": 2.1186,
+      "num_input_tokens_seen": 1663456,
+      "step": 145
+    },
+    {
+      "epoch": 1.1310084825636193,
+      "grad_norm": 3.5244438648223877,
+      "learning_rate": 4.570373196778427e-05,
+      "loss": 2.0526,
+      "num_input_tokens_seen": 1722320,
+      "step": 150
+    },
+    {
+      "epoch": 1.1687087653157398,
+      "grad_norm": 4.0647783279418945,
+      "learning_rate": 4.5421760615808474e-05,
+      "loss": 2.1589,
+      "num_input_tokens_seen": 1771664,
+      "step": 155
+    },
+    {
+      "epoch": 1.2064090480678604,
+      "grad_norm": 3.4753596782684326,
+      "learning_rate": 4.513175643827647e-05,
+      "loss": 2.0505,
+      "num_input_tokens_seen": 1829824,
+      "step": 160
+    },
+    {
+      "epoch": 1.244109330819981,
+      "grad_norm": 3.573631525039673,
+      "learning_rate": 4.4833833507280884e-05,
+      "loss": 2.0924,
+      "num_input_tokens_seen": 1882384,
+      "step": 165
+    },
+    {
+      "epoch": 1.2818096135721018,
+      "grad_norm": 3.39180588722229,
+      "learning_rate": 4.4528109009727336e-05,
+      "loss": 2.0184,
+      "num_input_tokens_seen": 1939456,
+      "step": 170
+    },
+    {
+      "epoch": 1.3195098963242224,
+      "grad_norm": 4.136682987213135,
+      "learning_rate": 4.42147032012394e-05,
+      "loss": 2.1524,
+      "num_input_tokens_seen": 1996320,
+      "step": 175
+    },
+    {
+      "epoch": 1.3572101790763431,
+      "grad_norm": 3.9565956592559814,
+      "learning_rate": 4.389373935885646e-05,
+      "loss": 2.1253,
+      "num_input_tokens_seen": 2056560,
+      "step": 180
+    },
+    {
+      "epoch": 1.3949104618284638,
+      "grad_norm": 3.7363975048065186,
+      "learning_rate": 4.356534373254316e-05,
+      "loss": 2.0752,
+      "num_input_tokens_seen": 2118496,
+      "step": 185
+    },
+    {
+      "epoch": 1.4326107445805842,
+      "grad_norm": 4.311358451843262,
+      "learning_rate": 4.322964549552943e-05,
+      "loss": 2.0574,
+      "num_input_tokens_seen": 2178352,
+      "step": 190
+    },
+    {
+      "epoch": 1.4703110273327051,
+      "grad_norm": 3.684499502182007,
+      "learning_rate": 4.288677669350066e-05,
+      "loss": 2.0481,
+      "num_input_tokens_seen": 2233440,
+      "step": 195
+    },
+    {
+      "epoch": 1.5080113100848256,
+      "grad_norm": 4.71754264831543,
+      "learning_rate": 4.2536872192658036e-05,
+      "loss": 2.0859,
+      "num_input_tokens_seen": 2289648,
+      "step": 200
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 792,
+  "num_input_tokens_seen": 2289648,
+  "num_train_epochs": 6,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9.797355635631718e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71c7be1d061ec911f0c413e914d833bd67f4314dfb89be07d4786b40787b7e69
+size 5432

checkpoint-300/README.md

@@ -0,0 +1,202 @@
+---
+base_model: alexsherstinsky/Mistral-7B-v0.1-sharded
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-300/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "alexsherstinsky/Mistral-7B-v0.1-sharded",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "q_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:47f32352835d32f789c14df740d30485836de2a1db3ea4b872c964197d5ca22d
+size 83945296

checkpoint-300/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58327d5c1f654db631a9b390523be73128b8929da5aa50284c18f3a3f8bfb2ec
+size 168149074

checkpoint-300/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34bcae41c589c7e4cab7b2ef263b878c90c2741404a6af11994dc31537b2319b
+size 14512

checkpoint-300/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d05dc84075e8f7dd1191c36f3be9dda12073208e12f7d2cef433c38d6336774a
+size 14512

checkpoint-300/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8ec54f6a0aea46713028de7bebd175e905179497901d87a9a354db5ce43f81a
+size 1064

checkpoint-300/special_tokens_map.json

@@ -0,0 +1,29 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-300/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-300/tokenizer_config.json

@@ -0,0 +1,50 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{{ '<s>' }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}{% if system_message is defined %}{{ system_message }}{% endif %}{% for message in loop_messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '[INST] ' + content + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ content + '</s>' }}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

checkpoint-300/trainer_state.json

@@ -0,0 +1,513 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.2620169651272386,
+  "eval_steps": 500,
+  "global_step": 300,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03770028275212064,
+      "grad_norm": 2.8582637310028076,
+      "learning_rate": 4.9995083170283816e-05,
+      "loss": 2.995,
+      "num_input_tokens_seen": 57008,
+      "step": 5
+    },
+    {
+      "epoch": 0.07540056550424128,
+      "grad_norm": 2.4749696254730225,
+      "learning_rate": 4.998033461515242e-05,
+      "loss": 2.7127,
+      "num_input_tokens_seen": 117936,
+      "step": 10
+    },
+    {
+      "epoch": 0.11310084825636192,
+      "grad_norm": 2.3738160133361816,
+      "learning_rate": 4.9955760135896534e-05,
+      "loss": 2.6302,
+      "num_input_tokens_seen": 176288,
+      "step": 15
+    },
+    {
+      "epoch": 0.15080113100848255,
+      "grad_norm": 2.2132370471954346,
+      "learning_rate": 4.992136939879856e-05,
+      "loss": 2.5818,
+      "num_input_tokens_seen": 229712,
+      "step": 20
+    },
+    {
+      "epoch": 0.1885014137606032,
+      "grad_norm": 2.444084882736206,
+      "learning_rate": 4.9877175931330346e-05,
+      "loss": 2.4913,
+      "num_input_tokens_seen": 288784,
+      "step": 25
+    },
+    {
+      "epoch": 0.22620169651272384,
+      "grad_norm": 2.190948009490967,
+      "learning_rate": 4.982319711683221e-05,
+      "loss": 2.4694,
+      "num_input_tokens_seen": 347712,
+      "step": 30
+    },
+    {
+      "epoch": 0.2639019792648445,
+      "grad_norm": 1.9827752113342285,
+      "learning_rate": 4.975945418767529e-05,
+      "loss": 2.4525,
+      "num_input_tokens_seen": 404272,
+      "step": 35
+    },
+    {
+      "epoch": 0.3016022620169651,
+      "grad_norm": 2.3744397163391113,
+      "learning_rate": 4.968597221690986e-05,
+      "loss": 2.4745,
+      "num_input_tokens_seen": 461632,
+      "step": 40
+    },
+    {
+      "epoch": 0.3393025447690858,
+      "grad_norm": 2.5817763805389404,
+      "learning_rate": 4.96027801084029e-05,
+      "loss": 2.4637,
+      "num_input_tokens_seen": 518016,
+      "step": 45
+    },
+    {
+      "epoch": 0.3770028275212064,
+      "grad_norm": 2.2218539714813232,
+      "learning_rate": 4.950991058546893e-05,
+      "loss": 2.5011,
+      "num_input_tokens_seen": 579312,
+      "step": 50
+    },
+    {
+      "epoch": 0.41470311027332707,
+      "grad_norm": 2.5139260292053223,
+      "learning_rate": 4.940740017799833e-05,
+      "loss": 2.4397,
+      "num_input_tokens_seen": 636448,
+      "step": 55
+    },
+    {
+      "epoch": 0.4524033930254477,
+      "grad_norm": 2.589820623397827,
+      "learning_rate": 4.929528920808854e-05,
+      "loss": 2.4109,
+      "num_input_tokens_seen": 693504,
+      "step": 60
+    },
+    {
+      "epoch": 0.49010367577756836,
+      "grad_norm": 2.8880276679992676,
+      "learning_rate": 4.917362177418342e-05,
+      "loss": 2.4228,
+      "num_input_tokens_seen": 752160,
+      "step": 65
+    },
+    {
+      "epoch": 0.527803958529689,
+      "grad_norm": 2.480609178543091,
+      "learning_rate": 4.904244573372733e-05,
+      "loss": 2.4391,
+      "num_input_tokens_seen": 811296,
+      "step": 70
+    },
+    {
+      "epoch": 0.5655042412818096,
+      "grad_norm": 2.6764094829559326,
+      "learning_rate": 4.8901812684340564e-05,
+      "loss": 2.4552,
+      "num_input_tokens_seen": 867696,
+      "step": 75
+    },
+    {
+      "epoch": 0.6032045240339302,
+      "grad_norm": 2.666821002960205,
+      "learning_rate": 4.8751777943523634e-05,
+      "loss": 2.3658,
+      "num_input_tokens_seen": 924944,
+      "step": 80
+    },
+    {
+      "epoch": 0.6409048067860509,
+      "grad_norm": 2.884999990463257,
+      "learning_rate": 4.8592400526898314e-05,
+      "loss": 2.359,
+      "num_input_tokens_seen": 978768,
+      "step": 85
+    },
+    {
+      "epoch": 0.6786050895381716,
+      "grad_norm": 2.58659291267395,
+      "learning_rate": 4.842374312499405e-05,
+      "loss": 2.4228,
+      "num_input_tokens_seen": 1034544,
+      "step": 90
+    },
+    {
+      "epoch": 0.7163053722902922,
+      "grad_norm": 2.6272594928741455,
+      "learning_rate": 4.824587207858888e-05,
+      "loss": 2.32,
+      "num_input_tokens_seen": 1091776,
+      "step": 95
+    },
+    {
+      "epoch": 0.7540056550424128,
+      "grad_norm": 2.6761183738708496,
+      "learning_rate": 4.805885735261454e-05,
+      "loss": 2.3393,
+      "num_input_tokens_seen": 1153056,
+      "step": 100
+    },
+    {
+      "epoch": 0.7917059377945335,
+      "grad_norm": 2.721721887588501,
+      "learning_rate": 4.786277250863599e-05,
+      "loss": 2.2595,
+      "num_input_tokens_seen": 1211808,
+      "step": 105
+    },
+    {
+      "epoch": 0.8294062205466541,
+      "grad_norm": 2.718644618988037,
+      "learning_rate": 4.765769467591625e-05,
+      "loss": 2.3171,
+      "num_input_tokens_seen": 1273600,
+      "step": 110
+    },
+    {
+      "epoch": 0.8671065032987747,
+      "grad_norm": 2.6614394187927246,
+      "learning_rate": 4.744370452107789e-05,
+      "loss": 2.2949,
+      "num_input_tokens_seen": 1331392,
+      "step": 115
+    },
+    {
+      "epoch": 0.9048067860508954,
+      "grad_norm": 2.851165771484375,
+      "learning_rate": 4.722088621637309e-05,
+      "loss": 2.3818,
+      "num_input_tokens_seen": 1387664,
+      "step": 120
+    },
+    {
+      "epoch": 0.942507068803016,
+      "grad_norm": 2.692765235900879,
+      "learning_rate": 4.698932740657479e-05,
+      "loss": 2.2494,
+      "num_input_tokens_seen": 1445968,
+      "step": 125
+    },
+    {
+      "epoch": 0.9802073515551367,
+      "grad_norm": 2.7365992069244385,
+      "learning_rate": 4.6749119174501975e-05,
+      "loss": 2.3593,
+      "num_input_tokens_seen": 1497744,
+      "step": 130
+    },
+    {
+      "epoch": 1.0179076343072573,
+      "grad_norm": 2.6355655193328857,
+      "learning_rate": 4.6500356005192514e-05,
+      "loss": 2.1892,
+      "num_input_tokens_seen": 1549008,
+      "step": 135
+    },
+    {
+      "epoch": 1.055607917059378,
+      "grad_norm": 2.8112802505493164,
+      "learning_rate": 4.6243135748737864e-05,
+      "loss": 2.1753,
+      "num_input_tokens_seen": 1605152,
+      "step": 140
+    },
+    {
+      "epoch": 1.0933081998114986,
+      "grad_norm": 3.1792027950286865,
+      "learning_rate": 4.597755958179406e-05,
+      "loss": 2.1186,
+      "num_input_tokens_seen": 1663456,
+      "step": 145
+    },
+    {
+      "epoch": 1.1310084825636193,
+      "grad_norm": 3.5244438648223877,
+      "learning_rate": 4.570373196778427e-05,
+      "loss": 2.0526,
+      "num_input_tokens_seen": 1722320,
+      "step": 150
+    },
+    {
+      "epoch": 1.1687087653157398,
+      "grad_norm": 4.0647783279418945,
+      "learning_rate": 4.5421760615808474e-05,
+      "loss": 2.1589,
+      "num_input_tokens_seen": 1771664,
+      "step": 155
+    },
+    {
+      "epoch": 1.2064090480678604,
+      "grad_norm": 3.4753596782684326,
+      "learning_rate": 4.513175643827647e-05,
+      "loss": 2.0505,
+      "num_input_tokens_seen": 1829824,
+      "step": 160
+    },
+    {
+      "epoch": 1.244109330819981,
+      "grad_norm": 3.573631525039673,
+      "learning_rate": 4.4833833507280884e-05,
+      "loss": 2.0924,
+      "num_input_tokens_seen": 1882384,
+      "step": 165
+    },
+    {
+      "epoch": 1.2818096135721018,
+      "grad_norm": 3.39180588722229,
+      "learning_rate": 4.4528109009727336e-05,
+      "loss": 2.0184,
+      "num_input_tokens_seen": 1939456,
+      "step": 170
+    },
+    {
+      "epoch": 1.3195098963242224,
+      "grad_norm": 4.136682987213135,
+      "learning_rate": 4.42147032012394e-05,
+      "loss": 2.1524,
+      "num_input_tokens_seen": 1996320,
+      "step": 175
+    },
+    {
+      "epoch": 1.3572101790763431,
+      "grad_norm": 3.9565956592559814,
+      "learning_rate": 4.389373935885646e-05,
+      "loss": 2.1253,
+      "num_input_tokens_seen": 2056560,
+      "step": 180
+    },
+    {
+      "epoch": 1.3949104618284638,
+      "grad_norm": 3.7363975048065186,
+      "learning_rate": 4.356534373254316e-05,
+      "loss": 2.0752,
+      "num_input_tokens_seen": 2118496,
+      "step": 185
+    },
+    {
+      "epoch": 1.4326107445805842,
+      "grad_norm": 4.311358451843262,
+      "learning_rate": 4.322964549552943e-05,
+      "loss": 2.0574,
+      "num_input_tokens_seen": 2178352,
+      "step": 190
+    },
+    {
+      "epoch": 1.4703110273327051,
+      "grad_norm": 3.684499502182007,
+      "learning_rate": 4.288677669350066e-05,
+      "loss": 2.0481,
+      "num_input_tokens_seen": 2233440,
+      "step": 195
+    },
+    {
+      "epoch": 1.5080113100848256,
+      "grad_norm": 4.71754264831543,
+      "learning_rate": 4.2536872192658036e-05,
+      "loss": 2.0859,
+      "num_input_tokens_seen": 2289648,
+      "step": 200
+    },
+    {
+      "epoch": 1.5457115928369463,
+      "grad_norm": 4.160897731781006,
+      "learning_rate": 4.218006962666934e-05,
+      "loss": 2.0877,
+      "num_input_tokens_seen": 2346224,
+      "step": 205
+    },
+    {
+      "epoch": 1.583411875589067,
+      "grad_norm": 3.895419120788574,
+      "learning_rate": 4.181650934253132e-05,
+      "loss": 2.0488,
+      "num_input_tokens_seen": 2407392,
+      "step": 210
+    },
+    {
+      "epoch": 1.6211121583411876,
+      "grad_norm": 4.271753787994385,
+      "learning_rate": 4.144633434536467e-05,
+      "loss": 2.0074,
+      "num_input_tokens_seen": 2462400,
+      "step": 215
+    },
+    {
+      "epoch": 1.6588124410933083,
+      "grad_norm": 4.415102958679199,
+      "learning_rate": 4.1069690242163484e-05,
+      "loss": 2.0844,
+      "num_input_tokens_seen": 2518304,
+      "step": 220
+    },
+    {
+      "epoch": 1.6965127238454287,
+      "grad_norm": 4.525811672210693,
+      "learning_rate": 4.06867251845213e-05,
+      "loss": 2.0943,
+      "num_input_tokens_seen": 2583168,
+      "step": 225
+    },
+    {
+      "epoch": 1.7342130065975496,
+      "grad_norm": 4.545309543609619,
+      "learning_rate": 4.0297589810356165e-05,
+      "loss": 2.0858,
+      "num_input_tokens_seen": 2643552,
+      "step": 230
+    },
+    {
+      "epoch": 1.77191328934967,
+      "grad_norm": 4.468845844268799,
+      "learning_rate": 3.9902437184657784e-05,
+      "loss": 2.044,
+      "num_input_tokens_seen": 2705600,
+      "step": 235
+    },
+    {
+      "epoch": 1.8096135721017907,
+      "grad_norm": 4.4386138916015625,
+      "learning_rate": 3.9501422739279956e-05,
+      "loss": 2.1083,
+      "num_input_tokens_seen": 2765856,
+      "step": 240
+    },
+    {
+      "epoch": 1.8473138548539114,
+      "grad_norm": 3.7564873695373535,
+      "learning_rate": 3.909470421180201e-05,
+      "loss": 2.0431,
+      "num_input_tokens_seen": 2824192,
+      "step": 245
+    },
+    {
+      "epoch": 1.885014137606032,
+      "grad_norm": 4.3798418045043945,
+      "learning_rate": 3.8682441583483314e-05,
+      "loss": 2.0978,
+      "num_input_tokens_seen": 2880208,
+      "step": 250
+    },
+    {
+      "epoch": 1.9227144203581528,
+      "grad_norm": 4.044529914855957,
+      "learning_rate": 3.8264797016335205e-05,
+      "loss": 2.096,
+      "num_input_tokens_seen": 2939856,
+      "step": 255
+    },
+    {
+      "epoch": 1.9604147031102732,
+      "grad_norm": 4.19630241394043,
+      "learning_rate": 3.7841934789335164e-05,
+      "loss": 1.9965,
+      "num_input_tokens_seen": 2995552,
+      "step": 260
+    },
+    {
+      "epoch": 1.998114985862394,
+      "grad_norm": 4.908398151397705,
+      "learning_rate": 3.741402123380828e-05,
+      "loss": 2.1569,
+      "num_input_tokens_seen": 3055504,
+      "step": 265
+    },
+    {
+      "epoch": 2.0358152686145146,
+      "grad_norm": 4.20371150970459,
+      "learning_rate": 3.6981224668001424e-05,
+      "loss": 1.7904,
+      "num_input_tokens_seen": 3110576,
+      "step": 270
+    },
+    {
+      "epoch": 2.0735155513666355,
+      "grad_norm": 5.414203643798828,
+      "learning_rate": 3.654371533087586e-05,
+      "loss": 1.7416,
+      "num_input_tokens_seen": 3171440,
+      "step": 275
+    },
+    {
+      "epoch": 2.111215834118756,
+      "grad_norm": 5.226822853088379,
+      "learning_rate": 3.610166531514436e-05,
+      "loss": 1.7513,
+      "num_input_tokens_seen": 3219792,
+      "step": 280
+    },
+    {
+      "epoch": 2.1489161168708764,
+      "grad_norm": 5.410066604614258,
+      "learning_rate": 3.565524849957921e-05,
+      "loss": 1.7565,
+      "num_input_tokens_seen": 3275792,
+      "step": 285
+    },
+    {
+      "epoch": 2.1866163996229973,
+      "grad_norm": 5.878467082977295,
+      "learning_rate": 3.520464048061758e-05,
+      "loss": 1.7417,
+      "num_input_tokens_seen": 3333136,
+      "step": 290
+    },
+    {
+      "epoch": 2.2243166823751177,
+      "grad_norm": 5.918631553649902,
+      "learning_rate": 3.47500185032913e-05,
+      "loss": 1.7726,
+      "num_input_tokens_seen": 3390080,
+      "step": 295
+    },
+    {
+      "epoch": 2.2620169651272386,
+      "grad_norm": 6.060715675354004,
+      "learning_rate": 3.4291561391508185e-05,
+      "loss": 1.7763,
+      "num_input_tokens_seen": 3446208,
+      "step": 300
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 792,
+  "num_input_tokens_seen": 3446208,
+  "num_train_epochs": 6,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.4746251566671462e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-300/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71c7be1d061ec911f0c413e914d833bd67f4314dfb89be07d4786b40787b7e69
+size 5432

checkpoint-400/README.md

@@ -0,0 +1,202 @@
+---
+base_model: alexsherstinsky/Mistral-7B-v0.1-sharded
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

checkpoint-400/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "alexsherstinsky/Mistral-7B-v0.1-sharded",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "q_proj",
+    "o_proj",
+    "gate_proj",
+    "up_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-400/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c39eb441954d4847ce4f1c0f2f7cf9d0321d48b8bd7fdbace4f11842d70b8f4b
+size 83945296

checkpoint-400/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c13c091ecbb1912e902799176b37929e0fb41892923330101344ffb0008bbeef
+size 168149074

checkpoint-400/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9affc1541e7e94c18354d5173bc55400c5f07faf3d080c6d453d48e7a8d6ac3
+size 14512

checkpoint-400/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4748c3ebf0e4c051c58b92e4a8c5b87cdb39d55cfdc2aec81a1baef0f02fc113
+size 14512

checkpoint-400/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f22ca2b24be2724497925de4a88dc3d652a6dcb3cb655731486697897d39d9be
+size 1064