Hugging Face's logo

c-ho
/
xlm_roberta_large_test_linsearch_only_abstract

Text Classification
Transformers
Safetensors
xlm-roberta
Generated from Trainer
Model card Files Community
c-ho commited on
Commit
f052f5d
·
verified ·
1 Parent(s): 623c73f

xlm_roberta_large_test_linsearch_only_abstract

Browse files
Files changed (3) hide show
  1. README.md +22 -20
  2. model.safetensors +1 -1
  3. training_args.bin +1 -1
README.md CHANGED
@@ -18,11 +18,11 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [FacebookAI/xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 1.3145
22
- - Accuracy: 0.6068
23
- - F1 Macro: 0.5065
24
- - Precision Macro: 0.5451
25
- - Recall Macro: 0.5192
26
 
27
  ## Model description
28
 
@@ -41,30 +41,32 @@ More information needed
41
  ### Training hyperparameters
42
 
43
  The following hyperparameters were used during training:
44
- - learning_rate: 2e-05
45
  - train_batch_size: 4
46
  - eval_batch_size: 4
47
  - seed: 42
 
 
48
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
49
- - lr_scheduler_type: cosine
50
- - lr_scheduler_warmup_ratio: 0.2
51
  - num_epochs: 10
52
  - mixed_precision_training: Native AMP
53
 
54
  ### Training results
55
 
56
- | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Macro | Precision Macro | Recall Macro |
57
- |:-------------:|:-----:|:------:|:---------------:|:--------:|:--------:|:---------------:|:------------:|
58
- | 1.3712 | 1.0 | 19722 | 1.3145 | 0.6068 | 0.5065 | 0.5451 | 0.5192 |
59
- | 2.9865 | 2.0 | 39444 | 2.9838 | 0.1173 | 0.0075 | 0.0042 | 0.0357 |
60
- | 2.9634 | 3.0 | 59166 | 2.9901 | 0.1173 | 0.0075 | 0.0042 | 0.0357 |
61
- | 2.9854 | 4.0 | 78888 | 2.9841 | 0.1173 | 0.0075 | 0.0042 | 0.0357 |
62
- | 2.9924 | 5.0 | 98610 | 2.9790 | 0.1173 | 0.0075 | 0.0042 | 0.0357 |
63
- | 2.9527 | 6.0 | 118332 | 2.9768 | 0.1173 | 0.0075 | 0.0042 | 0.0357 |
64
- | 2.9734 | 7.0 | 138054 | 2.9729 | 0.1173 | 0.0075 | 0.0042 | 0.0357 |
65
- | 2.9759 | 8.0 | 157776 | 2.9723 | 0.1246 | 0.0079 | 0.0045 | 0.0357 |
66
- | 2.9605 | 9.0 | 177498 | 2.9701 | 0.1246 | 0.0079 | 0.0045 | 0.0357 |
67
- | 2.963 | 10.0 | 197220 | 2.9705 | 0.1246 | 0.0079 | 0.0045 | 0.0357 |
68
 
69
 
70
  ### Framework versions
 
18
 
19
  This model is a fine-tuned version of [FacebookAI/xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 1.3349
22
+ - Accuracy: 0.6504
23
+ - F1 Macro: 0.6037
24
+ - Precision Macro: 0.6113
25
+ - Recall Macro: 0.6008
26
 
27
  ## Model description
28
 
 
41
  ### Training hyperparameters
42
 
43
  The following hyperparameters were used during training:
44
+ - learning_rate: 3e-05
45
  - train_batch_size: 4
46
  - eval_batch_size: 4
47
  - seed: 42
48
+ - gradient_accumulation_steps: 4
49
+ - total_train_batch_size: 16
50
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
51
+ - lr_scheduler_type: linear
52
+ - lr_scheduler_warmup_ratio: 0.1
53
  - num_epochs: 10
54
  - mixed_precision_training: Native AMP
55
 
56
  ### Training results
57
 
58
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Macro | Precision Macro | Recall Macro |
59
+ |:-------------:|:------:|:-----:|:---------------:|:--------:|:--------:|:---------------:|:------------:|
60
+ | 1.2114 | 1.0 | 4931 | 1.2224 | 0.6245 | 0.5249 | 0.5555 | 0.5501 |
61
+ | 1.0885 | 2.0 | 9862 | 1.1036 | 0.6427 | 0.5596 | 0.6009 | 0.5591 |
62
+ | 0.9781 | 3.0 | 14793 | 1.0828 | 0.6491 | 0.5760 | 0.6188 | 0.5811 |
63
+ | 0.8621 | 4.0 | 19724 | 1.0956 | 0.6569 | 0.5979 | 0.6363 | 0.6014 |
64
+ | 0.7267 | 5.0 | 24655 | 1.0899 | 0.6626 | 0.5970 | 0.6088 | 0.5941 |
65
+ | 0.6066 | 6.0 | 29586 | 1.2078 | 0.6517 | 0.5928 | 0.6177 | 0.5848 |
66
+ | 0.4627 | 7.0 | 34517 | 1.3349 | 0.6504 | 0.6037 | 0.6113 | 0.6008 |
67
+ | 0.3238 | 8.0 | 39448 | 1.5315 | 0.6398 | 0.5951 | 0.6064 | 0.5891 |
68
+ | 0.224 | 9.0 | 44379 | 1.8234 | 0.6438 | 0.5936 | 0.5994 | 0.5897 |
69
+ | 0.149 | 9.9981 | 49300 | 2.0762 | 0.6442 | 0.6011 | 0.6024 | 0.6007 |
70
 
71
 
72
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9695a0a6e8a34d7a0fb42f0e4eacbb503c935401f56b0f36c5a7eb86c6c99ccd
3
  size 2239729372
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a3a44375273e5da60b0b5f01de3de0b308f24cc8c3bb53c8b6b49f2bb4ac1174
3
  size 2239729372
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6d7e7b453f7783c1b240149ebc52c3d466f0dde49dc21e070a4df8b23beeb527
3
  size 5304
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc9e67b02f21aa0d9c731e7e5ec7634ca550522f9ddd0fa3a16449c38e8757bb
3
  size 5304