Hugging Face's logo

statking
/
Meta-Llama-3-8B-Instruct-ORPO-QLoRA

PEFT
Safetensors
llama
alignment-handbook
trl
orpo
Generated from Trainer
Model card Files
xet
Community
statking commited on
Commit
d21fe82
·
verified ·
1 Parent(s): de4580b

Model save

Browse files
Files changed (5) hide show
  1. README.md +97 -0
  2. adapter_model.safetensors +1 -1
  3. all_results.json +9 -0
  4. train_results.json +9 -0
  5. trainer_state.json +0 -0
README.md ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama3
3
+ library_name: peft
4
+ tags:
5
+ - trl
6
+ - orpo
7
+ - generated_from_trainer
8
+ base_model: meta-llama/Meta-Llama-3-8B-Instruct
9
+ model-index:
10
+ - name: Meta-Llama-3-8B-Instruct-ORPO-QLoRA
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/statking/huggingface/runs/5h649ptl)
18
+ # Meta-Llama-3-8B-Instruct-ORPO-QLoRA
19
+
20
+ This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on an unknown dataset.
21
+ It achieves the following results on the evaluation set:
22
+ - Loss: 0.5734
23
+ - Rewards/chosen: -0.0085
24
+ - Rewards/rejected: -0.0105
25
+ - Rewards/accuracies: 0.6070
26
+ - Rewards/margins: 0.0020
27
+ - Logps/rejected: -1.0493
28
+ - Logps/chosen: -0.8470
29
+ - Logits/rejected: -0.2369
30
+ - Logits/chosen: -0.2320
31
+ - Nll Loss: 0.5669
32
+ - Log Odds Ratio: -0.6615
33
+ - Log Odds Chosen: 0.3168
34
+
35
+ ## Model description
36
+
37
+ More information needed
38
+
39
+ ## Intended uses & limitations
40
+
41
+ More information needed
42
+
43
+ ## Training and evaluation data
44
+
45
+ More information needed
46
+
47
+ ## Training procedure
48
+
49
+ ### Training hyperparameters
50
+
51
+ The following hyperparameters were used during training:
52
+ - learning_rate: 7e-06
53
+ - train_batch_size: 4
54
+ - eval_batch_size: 8
55
+ - seed: 42
56
+ - distributed_type: multi-GPU
57
+ - num_devices: 2
58
+ - gradient_accumulation_steps: 4
59
+ - total_train_batch_size: 32
60
+ - total_eval_batch_size: 16
61
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
62
+ - lr_scheduler_type: cosine
63
+ - lr_scheduler_warmup_ratio: 0.1
64
+ - num_epochs: 1
65
+
66
+ ### Training results
67
+
68
+ | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | Nll Loss | Log Odds Ratio | Log Odds Chosen |
69
+ |:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|:--------:|:--------------:|:---------------:|
70
+ | 0.8633 | 0.0524 | 100 | 0.7181 | -0.0135 | -0.0158 | 0.6060 | 0.0023 | -1.5779 | -1.3476 | -0.4503 | -0.4466 | 0.7126 | -0.6965 | 0.2913 |
71
+ | 0.7831 | 0.1048 | 200 | 0.6487 | -0.0105 | -0.0125 | 0.6140 | 0.0020 | -1.2499 | -1.0520 | -0.3621 | -0.3619 | 0.6432 | -0.6627 | 0.2691 |
72
+ | 0.7146 | 0.1572 | 300 | 0.6238 | -0.0102 | -0.0122 | 0.6140 | 0.0020 | -1.2194 | -1.0173 | -0.3196 | -0.3169 | 0.6181 | -0.6594 | 0.2790 |
73
+ | 0.7361 | 0.2096 | 400 | 0.6137 | -0.0100 | -0.0120 | 0.6140 | 0.0020 | -1.2012 | -1.0014 | -0.2841 | -0.2811 | 0.6078 | -0.6618 | 0.2770 |
74
+ | 0.7382 | 0.2620 | 500 | 0.6066 | -0.0099 | -0.0119 | 0.6120 | 0.0020 | -1.1884 | -0.9868 | -0.3023 | -0.2982 | 0.6006 | -0.6603 | 0.2812 |
75
+ | 0.7339 | 0.3143 | 600 | 0.6009 | -0.0097 | -0.0118 | 0.6100 | 0.0020 | -1.1751 | -0.9714 | -0.2544 | -0.2490 | 0.5948 | -0.6587 | 0.2859 |
76
+ | 0.7133 | 0.3667 | 700 | 0.5968 | -0.0096 | -0.0116 | 0.6070 | 0.0020 | -1.1590 | -0.9588 | -0.2830 | -0.2764 | 0.5906 | -0.6590 | 0.2828 |
77
+ | 0.6988 | 0.4191 | 800 | 0.5926 | -0.0095 | -0.0115 | 0.6070 | 0.0020 | -1.1491 | -0.9451 | -0.2817 | -0.2745 | 0.5864 | -0.6576 | 0.2898 |
78
+ | 0.7493 | 0.4715 | 900 | 0.5882 | -0.0093 | -0.0114 | 0.6080 | 0.0021 | -1.1357 | -0.9301 | -0.2547 | -0.2476 | 0.5820 | -0.6552 | 0.2952 |
79
+ | 0.7022 | 0.5239 | 1000 | 0.5842 | -0.0091 | -0.0111 | 0.6070 | 0.0020 | -1.1110 | -0.9090 | -0.2588 | -0.2514 | 0.5780 | -0.6569 | 0.2962 |
80
+ | 0.6805 | 0.5763 | 1100 | 0.5807 | -0.0089 | -0.0108 | 0.6020 | 0.0020 | -1.0833 | -0.8865 | -0.2590 | -0.2519 | 0.5744 | -0.6608 | 0.2937 |
81
+ | 0.6427 | 0.6287 | 1200 | 0.5780 | -0.0087 | -0.0107 | 0.6070 | 0.0020 | -1.0670 | -0.8682 | -0.2483 | -0.2430 | 0.5717 | -0.6609 | 0.3024 |
82
+ | 0.6762 | 0.6811 | 1300 | 0.5762 | -0.0086 | -0.0106 | 0.6070 | 0.0020 | -1.0576 | -0.8586 | -0.2376 | -0.2322 | 0.5698 | -0.6618 | 0.3069 |
83
+ | 0.6944 | 0.7335 | 1400 | 0.5750 | -0.0085 | -0.0105 | 0.6070 | 0.0020 | -1.0548 | -0.8542 | -0.2468 | -0.2420 | 0.5686 | -0.6609 | 0.3102 |
84
+ | 0.6695 | 0.7859 | 1500 | 0.5742 | -0.0085 | -0.0105 | 0.6080 | 0.0020 | -1.0505 | -0.8493 | -0.2426 | -0.2372 | 0.5678 | -0.6616 | 0.3135 |
85
+ | 0.7258 | 0.8382 | 1600 | 0.5738 | -0.0085 | -0.0105 | 0.6080 | 0.0020 | -1.0497 | -0.8485 | -0.2418 | -0.2371 | 0.5673 | -0.6619 | 0.3140 |
86
+ | 0.7193 | 0.8906 | 1700 | 0.5735 | -0.0085 | -0.0105 | 0.6050 | 0.0020 | -1.0499 | -0.8477 | -0.2403 | -0.2352 | 0.5671 | -0.6610 | 0.3162 |
87
+ | 0.7038 | 0.9430 | 1800 | 0.5734 | -0.0085 | -0.0105 | 0.6090 | 0.0020 | -1.0493 | -0.8471 | -0.2360 | -0.2311 | 0.5670 | -0.6615 | 0.3164 |
88
+ | 0.6723 | 0.9954 | 1900 | 0.5734 | -0.0085 | -0.0105 | 0.6070 | 0.0020 | -1.0493 | -0.8470 | -0.2369 | -0.2320 | 0.5669 | -0.6615 | 0.3168 |
89
+
90
+
91
+ ### Framework versions
92
+
93
+ - PEFT 0.11.1
94
+ - Transformers 4.41.0
95
+ - Pytorch 2.3.0+cu121
96
+ - Datasets 2.19.1
97
+ - Tokenizers 0.19.1
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8692aadd104ad8bd03d6a1de1a49167b4aad5e89ff00b9cd863eddef0c3613c9
3
  size 671150064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91c16e6bf66b6ca7c01a347eb5da225ef87a3dfbf8ea4771cf8b917efccba6e4
3
  size 671150064
all_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.999607072691552,
3
+ "total_flos": 0.0,
4
+ "train_loss": 0.9266155345884759,
5
+ "train_runtime": 40145.686,
6
+ "train_samples": 61080,
7
+ "train_samples_per_second": 1.521,
8
+ "train_steps_per_second": 0.048
9
+ }
train_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.999607072691552,
3
+ "total_flos": 0.0,
4
+ "train_loss": 0.9266155345884759,
5
+ "train_runtime": 40145.686,
6
+ "train_samples": 61080,
7
+ "train_samples_per_second": 1.521,
8
+ "train_steps_per_second": 0.048
9
+ }
trainer_state.json ADDED
The diff for this file is too large to render. See raw diff