Hugging Face's logo

Kimory-X
/
zephyr-7b-orpo-qlora-lr5e6-beta0.1

PEFT
Safetensors
mistral
alignment-handbook
trl
orpo
Generated from Trainer
Model card Files Community
Kimory-X commited on
Commit
7c8f9af
·
verified ·
1 Parent(s): 36155bb

Model save

Browse files
Files changed (4) hide show
  1. README.md +91 -0
  2. all_results.json +9 -0
  3. train_results.json +9 -0
  4. trainer_state.json +0 -0
README.md ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: mistralai/Mistral-7B-v0.1
4
+ tags:
5
+ - trl
6
+ - orpo
7
+ - generated_from_trainer
8
+ library_name: peft
9
+ model-index:
10
+ - name: zephyr-7b-orpo-qlora-lr5e6-beta0.1
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/1014579852qq-tsinghua-university/huggingface/runs/97o9oz6g)
18
+ # zephyr-7b-orpo-qlora-lr5e6-beta0.1
19
+
20
+ This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on an unknown dataset.
21
+ It achieves the following results on the evaluation set:
22
+ - Loss: 1.0573
23
+ - Rewards/chosen: -0.0746
24
+ - Rewards/rejected: -0.0911
25
+ - Rewards/accuracies: 0.6168
26
+ - Rewards/margins: 0.0165
27
+ - Logps/rejected: -0.9108
28
+ - Logps/chosen: -0.7456
29
+ - Logits/rejected: -2.2160
30
+ - Logits/chosen: -2.3146
31
+ - Nll Loss: 1.0059
32
+ - Log Odds Ratio: -0.6434
33
+ - Log Odds Chosen: 0.2827
34
+
35
+ ## Model description
36
+
37
+ More information needed
38
+
39
+ ## Intended uses & limitations
40
+
41
+ More information needed
42
+
43
+ ## Training and evaluation data
44
+
45
+ More information needed
46
+
47
+ ## Training procedure
48
+
49
+ ### Training hyperparameters
50
+
51
+ The following hyperparameters were used during training:
52
+ - learning_rate: 5e-06
53
+ - train_batch_size: 2
54
+ - eval_batch_size: 4
55
+ - seed: 42
56
+ - distributed_type: multi-GPU
57
+ - num_devices: 5
58
+ - gradient_accumulation_steps: 4
59
+ - total_train_batch_size: 40
60
+ - total_eval_batch_size: 20
61
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
62
+ - lr_scheduler_type: cosine
63
+ - lr_scheduler_warmup_ratio: 0.1
64
+ - num_epochs: 1
65
+
66
+ ### Training results
67
+
68
+ | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | Nll Loss | Log Odds Ratio | Log Odds Chosen |
69
+ |:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|:--------:|:--------------:|:---------------:|
70
+ | 1.2019 | 0.0723 | 100 | 1.1583 | -0.0797 | -0.0926 | 0.5842 | 0.0129 | -0.9264 | -0.7970 | -2.2900 | -2.3902 | 1.0948 | -0.6684 | 0.2079 |
71
+ | 1.1793 | 0.1446 | 200 | 1.1165 | -0.0775 | -0.0918 | 0.6060 | 0.0143 | -0.9181 | -0.7754 | -2.2619 | -2.3628 | 1.0550 | -0.6553 | 0.2332 |
72
+ | 1.1317 | 0.2170 | 300 | 1.0960 | -0.0758 | -0.0910 | 0.6060 | 0.0153 | -0.9104 | -0.7575 | -2.2513 | -2.3506 | 1.0367 | -0.6496 | 0.2630 |
73
+ | 1.1023 | 0.2893 | 400 | 1.0837 | -0.0758 | -0.0914 | 0.6087 | 0.0156 | -0.9140 | -0.7583 | -2.2417 | -2.3391 | 1.0275 | -0.6482 | 0.2625 |
74
+ | 1.1022 | 0.3616 | 500 | 1.0748 | -0.0751 | -0.0911 | 0.6168 | 0.0160 | -0.9110 | -0.7507 | -2.2335 | -2.3303 | 1.0204 | -0.6471 | 0.2753 |
75
+ | 1.1102 | 0.4339 | 600 | 1.0691 | -0.0746 | -0.0908 | 0.6114 | 0.0162 | -0.9078 | -0.7461 | -2.2155 | -2.3130 | 1.0153 | -0.6447 | 0.2813 |
76
+ | 1.0911 | 0.5062 | 700 | 1.0641 | -0.0746 | -0.0909 | 0.6114 | 0.0163 | -0.9089 | -0.7463 | -2.2246 | -2.3233 | 1.0116 | -0.6446 | 0.2801 |
77
+ | 1.0863 | 0.5786 | 800 | 1.0610 | -0.0745 | -0.0912 | 0.6168 | 0.0167 | -0.9119 | -0.7445 | -2.2159 | -2.3155 | 1.0091 | -0.6425 | 0.2909 |
78
+ | 1.099 | 0.6509 | 900 | 1.0589 | -0.0749 | -0.0914 | 0.6168 | 0.0165 | -0.9135 | -0.7485 | -2.2140 | -2.3129 | 1.0076 | -0.6436 | 0.2801 |
79
+ | 1.067 | 0.7232 | 1000 | 1.0580 | -0.0745 | -0.0907 | 0.6168 | 0.0162 | -0.9071 | -0.7447 | -2.2185 | -2.3171 | 1.0064 | -0.6446 | 0.2788 |
80
+ | 1.1264 | 0.7955 | 1100 | 1.0574 | -0.0748 | -0.0913 | 0.6141 | 0.0165 | -0.9129 | -0.7477 | -2.2184 | -2.3166 | 1.0061 | -0.6437 | 0.2809 |
81
+ | 1.0909 | 0.8678 | 1200 | 1.0573 | -0.0746 | -0.0911 | 0.6168 | 0.0165 | -0.9108 | -0.7458 | -2.2147 | -2.3135 | 1.0059 | -0.6437 | 0.2825 |
82
+ | 1.1175 | 0.9402 | 1300 | 1.0573 | -0.0746 | -0.0911 | 0.6168 | 0.0165 | -0.9108 | -0.7456 | -2.2160 | -2.3146 | 1.0059 | -0.6434 | 0.2827 |
83
+
84
+
85
+ ### Framework versions
86
+
87
+ - PEFT 0.10.0
88
+ - Transformers 4.43.1
89
+ - Pytorch 2.1.2+cu121
90
+ - Datasets 2.18.0
91
+ - Tokenizers 0.19.1
all_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.9994576026035075,
3
+ "total_flos": 0.0,
4
+ "train_loss": 1.1206706619124682,
5
+ "train_runtime": 8808.4081,
6
+ "train_samples": 55305,
7
+ "train_samples_per_second": 6.279,
8
+ "train_steps_per_second": 0.157
9
+ }
train_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.9994576026035075,
3
+ "total_flos": 0.0,
4
+ "train_loss": 1.1206706619124682,
5
+ "train_runtime": 8808.4081,
6
+ "train_samples": 55305,
7
+ "train_samples_per_second": 6.279,
8
+ "train_steps_per_second": 0.157
9
+ }
trainer_state.json ADDED
The diff for this file is too large to render. See raw diff