Hugging Face's logo

pkarypis
/
gpt2-sft-port

Text Generation
Transformers
TensorBoard
Safetensors
llama
alignment-handbook
trl
sft
Generated from Trainer
conversational
text-generation-inference
Model card Files Metrics Community
pkarypis commited on
Commit
008f90f
·
verified ·
1 Parent(s): ec029e6

Model save

Browse files
Files changed (7) hide show
  1. README.md +8 -12
  2. all_results.json +4 -9
  3. model.safetensors +1 -1
  4. runs/Apr24_16-39-01_aga39/events.out.tfevents.1713994753.aga39.645447.0 +3 -0
  5. train_results.json +4 -4
  6. trainer_state.json +0 -0
  7. training_args.bin +1 -1
README.md CHANGED
@@ -2,15 +2,11 @@
2
  license: apache-2.0
3
  base_model: JackFram/llama-68m
4
  tags:
5
- - alignment-handbook
6
- - trl
7
- - sft
8
- - generated_from_trainer
9
  - trl
10
  - sft
11
  - generated_from_trainer
12
  datasets:
13
- - HuggingFaceH4/ultrachat_200k
14
  model-index:
15
  - name: gpt2-sft-port
16
  results: []
@@ -21,9 +17,9 @@ should probably proofread and complete it, then remove this comment. -->
21
 
22
  # gpt2-sft-port
23
 
24
- This model is a fine-tuned version of [JackFram/llama-68m](https://huggingface.co/JackFram/llama-68m) on the HuggingFaceH4/ultrachat_200k dataset.
25
  It achieves the following results on the evaluation set:
26
- - Loss: 2.0739
27
 
28
  ## Model description
29
 
@@ -47,9 +43,9 @@ The following hyperparameters were used during training:
47
  - eval_batch_size: 8
48
  - seed: 42
49
  - distributed_type: multi-GPU
50
- - num_devices: 4
51
- - total_train_batch_size: 128
52
- - total_eval_batch_size: 32
53
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
54
  - lr_scheduler_type: cosine
55
  - lr_scheduler_warmup_ratio: 0.1
@@ -59,8 +55,8 @@ The following hyperparameters were used during training:
59
 
60
  | Training Loss | Epoch | Step | Validation Loss |
61
  |:-------------:|:-----:|:----:|:---------------:|
62
- | 2.1085 | 1.0 | 2258 | 2.0943 |
63
- | 2.0651 | 2.0 | 4516 | 2.0739 |
64
 
65
 
66
  ### Framework versions
 
2
  license: apache-2.0
3
  base_model: JackFram/llama-68m
4
  tags:
 
 
 
 
5
  - trl
6
  - sft
7
  - generated_from_trainer
8
  datasets:
9
+ - generator
10
  model-index:
11
  - name: gpt2-sft-port
12
  results: []
 
17
 
18
  # gpt2-sft-port
19
 
20
+ This model is a fine-tuned version of [JackFram/llama-68m](https://huggingface.co/JackFram/llama-68m) on the generator dataset.
21
  It achieves the following results on the evaluation set:
22
+ - Loss: 2.1067
23
 
24
  ## Model description
25
 
 
43
  - eval_batch_size: 8
44
  - seed: 42
45
  - distributed_type: multi-GPU
46
+ - num_devices: 8
47
+ - total_train_batch_size: 256
48
+ - total_eval_batch_size: 64
49
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
50
  - lr_scheduler_type: cosine
51
  - lr_scheduler_warmup_ratio: 0.1
 
55
 
56
  | Training Loss | Epoch | Step | Validation Loss |
57
  |:-------------:|:-----:|:----:|:---------------:|
58
+ | 2.1213 | 1.0 | 1129 | 2.1273 |
59
+ | 2.0929 | 2.0 | 2258 | 2.1067 |
60
 
61
 
62
  ### Framework versions
all_results.json CHANGED
@@ -1,13 +1,8 @@
1
  {
2
  "epoch": 2.0,
3
- "eval_loss": 2.073899507522583,
4
- "eval_runtime": 26.2955,
5
- "eval_samples": 23110,
6
- "eval_samples_per_second": 1216.139,
7
- "eval_steps_per_second": 38.029,
8
- "train_loss": 2.136630948372284,
9
- "train_runtime": 1210.2491,
10
  "train_samples": 207865,
11
- "train_samples_per_second": 477.469,
12
- "train_steps_per_second": 3.731
13
  }
 
1
  {
2
  "epoch": 2.0,
3
+ "train_loss": 2.17739642542796,
4
+ "train_runtime": 641.6533,
 
 
 
 
 
5
  "train_samples": 207865,
6
+ "train_samples_per_second": 900.574,
7
+ "train_steps_per_second": 3.519
8
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3bd53bfc8f6bed8c963823ac774ec0d2e33cb8782ed7363cb8084ed925fed564
3
  size 136062744
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:05162d09558019270998ed5c5fe53121ea8e093b25ac1a9d4becaccf5356e6b6
3
  size 136062744
runs/Apr24_16-39-01_aga39/events.out.tfevents.1713994753.aga39.645447.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c5539baaaf5023aef2f3538e240adf4cafe1de9c7315aa80d1e471c8f7caf719
3
+ size 100878
train_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
  "epoch": 2.0,
3
- "train_loss": 2.136630948372284,
4
- "train_runtime": 1210.2491,
5
  "train_samples": 207865,
6
- "train_samples_per_second": 477.469,
7
- "train_steps_per_second": 3.731
8
  }
 
1
  {
2
  "epoch": 2.0,
3
+ "train_loss": 2.17739642542796,
4
+ "train_runtime": 641.6533,
5
  "train_samples": 207865,
6
+ "train_samples_per_second": 900.574,
7
+ "train_steps_per_second": 3.519
8
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff
 
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a17dccee37de884ea4e6472fc2742ff9af92f2cc68f9ac51f033e585d3744a2a
3
  size 6072
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b265d5a384c5c1ec6439688f515216fcdb4a4736a63d3e75fbc4bf184c4bfc01
3
  size 6072