Hugging Face's logo

tabbas97
/
distilbert-base-uncased-finetuned-pubmed-lora-trained-tabbas97

PEFT
Safetensors
Generated from Trainer
Model card Files Community
tabbas97 commited on
Commit
7949fef
·
verified ·
1 Parent(s): 4c8ed3d

End of training

Browse files
Files changed (1) hide show
  1. README.md +22 -14
README.md CHANGED
@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the pubmed-summarization dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 1.9835
22
 
23
  ## Model description
24
 
@@ -38,7 +38,7 @@ More information needed
38
 
39
  The following hyperparameters were used during training:
40
  - learning_rate: 5e-05
41
- - train_batch_size: 8
42
  - eval_batch_size: 8
43
  - seed: 42
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
@@ -48,18 +48,26 @@ The following hyperparameters were used during training:
48
 
49
  ### Training results
50
 
51
- | Training Loss | Epoch | Step | Validation Loss |
52
- |:-------------:|:-----:|:----:|:---------------:|
53
- | 2.2356 | 0.4 | 500 | 2.0414 |
54
- | 2.161 | 0.8 | 1000 | 2.0307 |
55
- | 2.1446 | 1.2 | 1500 | 1.9946 |
56
- | 2.143 | 1.6 | 2000 | 2.0254 |
57
- | 2.1318 | 2.0 | 2500 | 1.9951 |
58
- | 2.133 | 2.4 | 3000 | 2.0143 |
59
- | 2.1321 | 2.8 | 3500 | 1.9991 |
60
- | 2.1268 | 3.2 | 4000 | 1.9789 |
61
- | 2.1169 | 3.6 | 4500 | 1.9736 |
62
- | 2.1254 | 4.0 | 5000 | 1.9745 |
 
 
 
 
 
 
 
 
63
 
64
 
65
  ### Framework versions
 
18
 
19
  This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the pubmed-summarization dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 1.9256
22
 
23
  ## Model description
24
 
 
38
 
39
  The following hyperparameters were used during training:
40
  - learning_rate: 5e-05
41
+ - train_batch_size: 32
42
  - eval_batch_size: 8
43
  - seed: 42
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 
48
 
49
  ### Training results
50
 
51
+ | Training Loss | Epoch | Step | Validation Loss |
52
+ |:-------------:|:------:|:----:|:---------------:|
53
+ | 2.1986 | 0.1667 | 500 | 2.0156 |
54
+ | 2.1414 | 0.3334 | 1000 | 1.9893 |
55
+ | 2.1247 | 0.5002 | 1500 | 1.9770 |
56
+ | 2.1106 | 0.6669 | 2000 | 1.9640 |
57
+ | 2.103 | 0.8336 | 2500 | 1.9548 |
58
+ | 2.0974 | 1.0003 | 3000 | 1.9519 |
59
+ | 2.0874 | 1.1671 | 3500 | 1.9506 |
60
+ | 2.0842 | 1.3338 | 4000 | 1.9470 |
61
+ | 2.0799 | 1.5005 | 4500 | 1.9406 |
62
+ | 2.0781 | 1.6672 | 5000 | 1.9363 |
63
+ | 2.0763 | 1.8339 | 5500 | 1.9371 |
64
+ | 2.0664 | 2.0007 | 6000 | 1.9311 |
65
+ | 2.0717 | 2.1674 | 6500 | 1.9277 |
66
+ | 2.0683 | 2.3341 | 7000 | 1.9247 |
67
+ | 2.0622 | 2.5008 | 7500 | 1.9290 |
68
+ | 2.0614 | 2.6676 | 8000 | 1.9170 |
69
+ | 2.0614 | 2.8343 | 8500 | 1.9239 |
70
+ | 2.0646 | 3.0010 | 9000 | 1.9211 |
71
 
72
 
73
  ### Framework versions