Hugging Face's logo

terry69
/
zephyr-7b-sft-qlora

PEFT
TensorBoard
Safetensors
mistral
trl
sft
Generated from Trainer
Model card Files Metrics Community
terry69 commited on
Commit
f77ffbc
·
verified ·
1 Parent(s): 1eb09f1

Model save

Browse files
Files changed (6) hide show
  1. README.md +4 -6
  2. adapter_model.safetensors +1 -1
  3. all_results.json +4 -4
  4. runs/Mar15_22-41-49_ip-172-31-69-60.ec2.internal/events.out.tfevents.1710543612.ip-172-31-69-60.ec2.internal.38124.0 +2 -2
  5. train_results.json +4 -4
  6. trainer_state.json +0 -0
README.md CHANGED
@@ -2,14 +2,12 @@
2
  license: apache-2.0
3
  library_name: peft
4
  tags:
5
- - alignment-handbook
6
- - generated_from_trainer
7
  - trl
8
  - sft
9
  - generated_from_trainer
10
  base_model: mistralai/Mistral-7B-v0.1
11
  datasets:
12
- - HuggingFaceH4/ultrachat_200k
13
  model-index:
14
  - name: zephyr-7b-sft-qlora
15
  results: []
@@ -20,9 +18,9 @@ should probably proofread and complete it, then remove this comment. -->
20
 
21
  # zephyr-7b-sft-qlora
22
 
23
- This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the HuggingFaceH4/ultrachat_200k dataset.
24
  It achieves the following results on the evaluation set:
25
- - Loss: 1.0681
26
 
27
  ## Model description
28
 
@@ -59,7 +57,7 @@ The following hyperparameters were used during training:
59
 
60
  | Training Loss | Epoch | Step | Validation Loss |
61
  |:-------------:|:-----:|:-----:|:---------------:|
62
- | 1.0163 | 1.0 | 12485 | 1.0681 |
63
 
64
 
65
  ### Framework versions
 
2
  license: apache-2.0
3
  library_name: peft
4
  tags:
 
 
5
  - trl
6
  - sft
7
  - generated_from_trainer
8
  base_model: mistralai/Mistral-7B-v0.1
9
  datasets:
10
+ - generator
11
  model-index:
12
  - name: zephyr-7b-sft-qlora
13
  results: []
 
18
 
19
  # zephyr-7b-sft-qlora
20
 
21
+ This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the generator dataset.
22
  It achieves the following results on the evaluation set:
23
+ - Loss: 4.8866
24
 
25
  ## Model description
26
 
 
57
 
58
  | Training Loss | Epoch | Step | Validation Loss |
59
  |:-------------:|:-----:|:-----:|:---------------:|
60
+ | 4.8496 | 1.0 | 12498 | 4.8866 |
61
 
62
 
63
  ### Framework versions
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:df2f5026803d11f1c0741b6c6ed1d98b6566701782cff36f82a4b14f044e94c1
3
  size 42002584
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b828bbfb5006e45e584885b635ffcb70ea3683762c90cccff03b4be10522de5
3
  size 42002584
all_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "epoch": 1.0,
3
- "train_loss": 1.0040549779730794,
4
- "train_runtime": 174545.1979,
5
  "train_samples": 228651,
6
- "train_samples_per_second": 1.144,
7
- "train_steps_per_second": 0.072
8
  }
 
1
  {
2
  "epoch": 1.0,
3
+ "train_loss": 4.727149732982012,
4
+ "train_runtime": 175008.638,
5
  "train_samples": 228651,
6
+ "train_samples_per_second": 1.143,
7
+ "train_steps_per_second": 0.071
8
  }
runs/Mar15_22-41-49_ip-172-31-69-60.ec2.internal/events.out.tfevents.1710543612.ip-172-31-69-60.ec2.internal.38124.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8859f333b1b949a85e63462efcebf49f1d5d00d2d6d2cd788f7adcef3d415251
3
- size 371906
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:03a3a85686f891c8a3043d0075715753ee3e928c2d7418eda7eb09fb998c9514
3
+ size 397494
train_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "epoch": 1.0,
3
- "train_loss": 1.0040549779730794,
4
- "train_runtime": 174545.1979,
5
  "train_samples": 228651,
6
- "train_samples_per_second": 1.144,
7
- "train_steps_per_second": 0.072
8
  }
 
1
  {
2
  "epoch": 1.0,
3
+ "train_loss": 4.727149732982012,
4
+ "train_runtime": 175008.638,
5
  "train_samples": 228651,
6
+ "train_samples_per_second": 1.143,
7
+ "train_steps_per_second": 0.071
8
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff