Hugging Face's logo

aiface
/
xlm-roberta-large_massive_crf_v1

Transformers
Safetensors
Generated from Trainer
Model card Files
xet
Community
aiface commited on
Commit
1da2842
·
verified ·
1 Parent(s): 755a1b3

Model save

Browse files
Files changed (5) hide show
  1. README.md +85 -0
  2. intent_report_test.txt +75 -0
  3. model.safetensors +1 -1
  4. model_predict_test.csv +0 -0
  5. slot_report_test.txt +60 -0
README.md ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: mit
4
+ base_model: FacebookAI/xlm-roberta-large
5
+ tags:
6
+ - generated_from_trainer
7
+ model-index:
8
+ - name: xlm-roberta-large_massive_crf_v1
9
+ results: []
10
+ ---
11
+
12
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
+ should probably proofread and complete it, then remove this comment. -->
14
+
15
+ # xlm-roberta-large_massive_crf_v1
16
+
17
+ This model is a fine-tuned version of [FacebookAI/xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large) on an unknown dataset.
18
+ It achieves the following results on the evaluation set:
19
+ - Loss: 5.3967
20
+ - Slot P: 0.7375
21
+ - Slot R: 0.7801
22
+ - Slot F1: 0.7582
23
+ - Slot Exact Match: 0.7260
24
+ - Intent Acc: 0.8687
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 5e-05
44
+ - train_batch_size: 128
45
+ - eval_batch_size: 128
46
+ - seed: 42
47
+ - gradient_accumulation_steps: 2
48
+ - total_train_batch_size: 256
49
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
50
+ - lr_scheduler_type: cosine
51
+ - lr_scheduler_warmup_ratio: 0.06
52
+ - num_epochs: 30
53
+ - mixed_precision_training: Native AMP
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | Slot P | Slot R | Slot F1 | Slot Exact Match | Intent Acc |
58
+ |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:-------:|:----------------:|:----------:|
59
+ | No log | 1.0 | 45 | 19.9807 | 0.0 | 0.0 | 0.0 | 0.3187 | 0.0649 |
60
+ | 84.6375 | 2.0 | 90 | 9.4999 | 0.5191 | 0.5 | 0.5094 | 0.4634 | 0.3453 |
61
+ | 26.6202 | 3.0 | 135 | 4.9217 | 0.6025 | 0.7060 | 0.6502 | 0.6119 | 0.7708 |
62
+ | 11.9124 | 4.0 | 180 | 4.0265 | 0.6567 | 0.7338 | 0.6931 | 0.6749 | 0.8411 |
63
+ | 7.1611 | 5.0 | 225 | 3.6002 | 0.6822 | 0.7647 | 0.7211 | 0.6985 | 0.8569 |
64
+ | 5.4223 | 6.0 | 270 | 3.6653 | 0.7199 | 0.7687 | 0.7435 | 0.7191 | 0.8578 |
65
+ | 4.1914 | 7.0 | 315 | 3.6212 | 0.7156 | 0.7711 | 0.7423 | 0.7201 | 0.8628 |
66
+ | 3.4499 | 8.0 | 360 | 3.9382 | 0.7019 | 0.7836 | 0.7405 | 0.7078 | 0.8662 |
67
+ | 2.8646 | 9.0 | 405 | 4.0638 | 0.7059 | 0.7856 | 0.7436 | 0.7093 | 0.8652 |
68
+ | 2.3159 | 10.0 | 450 | 4.1920 | 0.7117 | 0.7751 | 0.7421 | 0.7127 | 0.8682 |
69
+ | 2.3159 | 11.0 | 495 | 4.3891 | 0.7110 | 0.7736 | 0.7410 | 0.7142 | 0.8731 |
70
+ | 1.8513 | 12.0 | 540 | 4.4429 | 0.7295 | 0.7821 | 0.7549 | 0.7231 | 0.8741 |
71
+ | 1.6003 | 13.0 | 585 | 4.7107 | 0.7317 | 0.7841 | 0.7570 | 0.7211 | 0.8775 |
72
+ | 1.3617 | 14.0 | 630 | 4.8732 | 0.7311 | 0.7751 | 0.7525 | 0.7231 | 0.8721 |
73
+ | 1.0467 | 15.0 | 675 | 5.0702 | 0.7230 | 0.7816 | 0.7511 | 0.7211 | 0.8687 |
74
+ | 0.9095 | 16.0 | 720 | 5.1884 | 0.7377 | 0.7935 | 0.7646 | 0.7304 | 0.8775 |
75
+ | 0.7422 | 17.0 | 765 | 5.1458 | 0.7267 | 0.7831 | 0.7538 | 0.7182 | 0.8711 |
76
+ | 0.6169 | 18.0 | 810 | 5.5018 | 0.7326 | 0.7836 | 0.7572 | 0.7231 | 0.8711 |
77
+ | 0.5221 | 19.0 | 855 | 5.3967 | 0.7375 | 0.7801 | 0.7582 | 0.7260 | 0.8687 |
78
+
79
+
80
+ ### Framework versions
81
+
82
+ - Transformers 4.55.0
83
+ - Pytorch 2.7.0+cu126
84
+ - Datasets 3.6.0
85
+ - Tokenizers 0.21.4
intent_report_test.txt ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ precision recall f1-score support
2
+
3
+ 0 0.91 0.97 0.94 88
4
+ 1 0.84 0.89 0.86 36
5
+ 2 0.95 1.00 0.97 35
6
+ 3 0.93 0.80 0.86 35
7
+ 4 0.89 0.92 0.91 26
8
+ 5 0.00 0.00 0.00 1
9
+ 6 0.71 0.81 0.76 43
10
+ 7 0.75 0.75 0.75 4
11
+ 8 1.00 1.00 1.00 18
12
+ 9 0.94 0.90 0.92 72
13
+ 10 0.97 1.00 0.99 39
14
+ 11 0.71 1.00 0.83 15
15
+ 12 0.71 0.60 0.65 169
16
+ 13 0.95 0.96 0.95 156
17
+ 14 0.86 0.92 0.89 13
18
+ 15 0.58 0.92 0.71 12
19
+ 16 0.68 0.95 0.79 22
20
+ 17 0.69 0.92 0.79 26
21
+ 18 0.91 0.78 0.84 27
22
+ 19 0.77 0.87 0.82 31
23
+ 20 0.84 0.88 0.86 41
24
+ 21 0.90 0.90 0.90 39
25
+ 22 0.83 0.86 0.85 124
26
+ 23 1.00 0.91 0.95 34
27
+ 24 1.00 0.90 0.95 10
28
+ 25 1.00 1.00 1.00 19
29
+ 26 0.91 0.88 0.89 57
30
+ 27 0.87 0.80 0.83 25
31
+ 28 0.25 0.33 0.29 6
32
+ 29 1.00 0.50 0.67 6
33
+ 30 0.94 0.94 0.94 67
34
+ 31 0.94 0.81 0.87 21
35
+ 32 0.79 0.82 0.80 126
36
+ 33 0.97 0.93 0.95 114
37
+ 34 1.00 0.88 0.94 26
38
+ 35 0.92 1.00 0.96 11
39
+ 36 0.85 0.89 0.87 72
40
+ 37 0.00 0.00 0.00 0
41
+ 38 0.92 0.80 0.86 15
42
+ 39 0.92 0.92 0.92 25
43
+ 40 0.95 0.98 0.97 43
44
+ 41 0.17 0.33 0.22 3
45
+ 42 0.92 0.90 0.91 51
46
+ 43 0.80 0.89 0.84 36
47
+ 44 0.98 0.92 0.95 119
48
+ 45 0.91 0.87 0.89 176
49
+ 46 0.89 0.97 0.93 32
50
+ 47 0.97 0.91 0.94 81
51
+ 48 0.98 0.98 0.98 41
52
+ 49 0.81 0.87 0.84 141
53
+ 50 0.95 0.92 0.93 209
54
+ 51 0.92 0.94 0.93 35
55
+ 52 1.00 1.00 1.00 21
56
+ 53 0.91 0.92 0.91 52
57
+ 54 0.92 1.00 0.96 23
58
+ 55 0.74 0.70 0.72 20
59
+ 56 1.00 0.97 0.99 36
60
+ 57 0.85 0.83 0.84 35
61
+ 58 0.91 0.84 0.88 63
62
+ 59 0.88 0.84 0.86 51
63
+
64
+ accuracy 0.88 2974
65
+ macro avg 0.83 0.84 0.83 2974
66
+ weighted avg 0.89 0.88 0.88 2974
67
+
68
+ Confusion matrix:
69
+ [[85 0 0 ... 0 0 0]
70
+ [ 0 32 0 ... 1 0 0]
71
+ [ 0 0 35 ... 0 0 0]
72
+ ...
73
+ [ 0 0 0 ... 29 0 0]
74
+ [ 0 0 0 ... 0 53 0]
75
+ [ 0 0 0 ... 0 1 43]]
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a8e55e856e5d6250dd87e640cbe11dfacfe76166f5626b5c4c1d918e3e267034
3
  size 2240362200
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2dedd304f9b7f8d37596277cc9d68b7c82f7bc4cc6cf9a1eb5d57b7ff1d22d10
3
  size 2240362200
model_predict_test.csv ADDED
The diff for this file is too large to render. See raw diff
 
slot_report_test.txt ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ precision recall f1-score support
2
+
3
+ alarm_type 0.40 1.00 0.57 2
4
+ app_name 0.36 1.00 0.53 5
5
+ artist_name 0.70 0.87 0.77 61
6
+ audiobook_author 1.00 0.80 0.89 5
7
+ audiobook_name 0.71 0.74 0.72 23
8
+ business_name 0.81 0.87 0.84 92
9
+ business_type 0.42 0.52 0.46 31
10
+ change_amount 0.60 0.67 0.63 9
11
+ coffee_type 0.60 0.75 0.67 4
12
+ color_type 0.53 0.73 0.61 26
13
+ cooking_type 0.56 0.62 0.59 8
14
+ currency_name 0.79 0.92 0.85 50
15
+ date 0.80 0.89 0.84 415
16
+ definition_word 0.85 0.88 0.87 51
17
+ device_type 0.70 0.75 0.73 57
18
+ drink_type 0.00 0.00 0.00 1
19
+ email_address 1.00 1.00 1.00 9
20
+ email_folder 0.57 0.80 0.67 5
21
+ event_name 0.63 0.69 0.66 260
22
+ food_type 0.62 0.74 0.67 72
23
+ game_name 0.86 0.92 0.89 26
24
+ general_frequency 0.57 0.65 0.60 20
25
+ house_place 0.84 0.88 0.86 58
26
+ ingredient 0.00 0.00 0.00 6
27
+ joke_type 0.83 0.91 0.87 11
28
+ list_name 0.79 0.74 0.76 61
29
+ meal_type 0.65 0.94 0.77 18
30
+ media_type 0.82 0.87 0.84 128
31
+ movie_name 0.25 0.50 0.33 2
32
+ movie_type 0.67 0.67 0.67 3
33
+ music_album 0.00 0.00 0.00 1
34
+ music_descriptor 0.11 0.29 0.16 7
35
+ music_genre 0.78 0.84 0.81 50
36
+ news_topic 0.42 0.48 0.45 52
37
+ order_type 0.56 0.90 0.69 20
38
+ person 0.77 0.87 0.82 216
39
+ personal_info 0.71 0.86 0.77 14
40
+ place_name 0.82 0.82 0.82 281
41
+ player_setting 0.53 0.57 0.55 40
42
+ playlist_name 0.31 0.27 0.29 15
43
+ podcast_descriptor 0.65 0.54 0.59 24
44
+ podcast_name 0.67 0.59 0.62 17
45
+ radio_name 0.66 0.70 0.68 33
46
+ relation 0.62 0.76 0.69 59
47
+ song_name 0.70 0.82 0.75 39
48
+ sport_type 0.00 0.00 0.00 0
49
+ time 0.70 0.72 0.71 191
50
+ time_zone 0.71 0.77 0.74 13
51
+ timeofday 0.67 0.70 0.68 60
52
+ transport_agency 0.90 1.00 0.95 9
53
+ transport_descriptor 0.25 0.50 0.33 2
54
+ transport_name 0.60 0.75 0.67 4
55
+ transport_type 0.84 0.86 0.85 65
56
+ weather_descriptor 0.70 0.78 0.74 82
57
+
58
+ micro avg 0.72 0.79 0.75 2813
59
+ macro avg 0.60 0.70 0.64 2813
60
+ weighted avg 0.73 0.79 0.75 2813