Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,7 @@ metrics:
|
|
| 11 |
|
| 12 |
|
| 13 |
This is [ruRoberta-large](https://huggingface.co/ai-forever/ruRoberta-large) model finetuned on [ru_go_emotions](https://huggingface.co/datasets/seara/ru_go_emotions)
|
| 14 |
-
dataset for multilabel classification. Model can be used to extract all emotions from text or detect certain emotions.
|
| 15 |
|
| 16 |
# Usage
|
| 17 |
Using model with Huggingface Transformers:
|
|
@@ -88,40 +88,40 @@ print(predict('У вас отличный сервис и лучший кофе
|
|
| 88 |
'''
|
| 89 |
```
|
| 90 |
|
| 91 |
-
# Eval results on test split of ru-go-emotions
|
| 92 |
|
| 93 |
-
precision recall f1-score support threshold
|
| 94 |
-
|
| 95 |
-
admiration 0.63 0.75 0.69 504 0.37
|
| 96 |
-
amusement 0.76 0.91 0.83 264 0.29
|
| 97 |
-
anger 0.47 0.32 0.38 198 0.29
|
| 98 |
-
annoyance 0.33 0.39 0.36 320 0.16
|
| 99 |
-
approval 0.27 0.58 0.37 351 0.14
|
| 100 |
-
caring 0.32 0.59 0.41 135 0.14
|
| 101 |
-
confusion 0.41 0.52 0.46 153 0.18
|
| 102 |
-
curiosity 0.45 0.73 0.55 284 0.35
|
| 103 |
-
desire 0.54 0.31 0.40 83 0.33
|
| 104 |
-
disappointment 0.31 0.34 0.33 151 0.22
|
| 105 |
-
disapproval 0.31 0.57 0.40 267 0.20
|
| 106 |
-
disgust 0.44 0.40 0.42 123 0.29
|
| 107 |
-
embarrassment 0.48 0.38 0.42 37 0.18
|
| 108 |
-
excitement 0.29 0.43 0.34 103 0.29
|
| 109 |
-
fear 0.56 0.78 0.65 78 0.24
|
| 110 |
-
gratitude 0.95 0.85 0.89 352 0.71
|
| 111 |
-
grief 0.03 0.33 0.05 6 0.02
|
| 112 |
-
joy 0.48 0.58 0.53 161 0.31
|
| 113 |
-
love 0.73 0.84 0.78 238 0.45
|
| 114 |
-
nervousness 0.24 0.48 0.32 23 0.06
|
| 115 |
-
optimism 0.57 0.54 0.56 186 0.18
|
| 116 |
-
pride 0.67 0.38 0.48 16 0.04
|
| 117 |
-
realization 0.18 0.31 0.23 145 0.08
|
| 118 |
-
relief 0.30 0.27 0.29 11 0.10
|
| 119 |
-
remorse 0.53 0.84 0.65 56 0.22
|
| 120 |
-
sadness 0.56 0.53 0.55 156 0.39
|
| 121 |
-
surprise 0.55 0.57 0.56 141 0.35
|
| 122 |
-
neutral 0.59 0.79 0.68 1787 0.24
|
| 123 |
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
|
| 13 |
This is [ruRoberta-large](https://huggingface.co/ai-forever/ruRoberta-large) model finetuned on [ru_go_emotions](https://huggingface.co/datasets/seara/ru_go_emotions)
|
| 14 |
+
dataset for multilabel classification. Model can be used to extract all emotions from text or detect certain emotions. Thresholds are selected on validation set by maximizing f1 macro over all labels.
|
| 15 |
|
| 16 |
# Usage
|
| 17 |
Using model with Huggingface Transformers:
|
|
|
|
| 88 |
'''
|
| 89 |
```
|
| 90 |
|
| 91 |
+
# Eval results on test split of ru-go-emotions
|
| 92 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
|
| 94 |
+
| |precision|recall|f1-score|support|threshold|
|
| 95 |
+
|--------------|---------|------|--------|-------|---------|
|
| 96 |
+
|admiration |0.63 |0.75 |0.69 |504 |0.37 |
|
| 97 |
+
|amusement |0.76 |0.91 |0.83 |264 |0.29 |
|
| 98 |
+
|anger |0.47 |0.32 |0.38 |198 |0.29 |
|
| 99 |
+
|annoyance |0.33 |0.39 |0.36 |320 |0.16 |
|
| 100 |
+
|approval |0.27 |0.58 |0.37 |351 |0.14 |
|
| 101 |
+
|caring |0.32 |0.59 |0.41 |135 |0.14 |
|
| 102 |
+
|confusion |0.41 |0.52 |0.46 |153 |0.18 |
|
| 103 |
+
|curiosity |0.45 |0.73 |0.55 |284 |0.35 |
|
| 104 |
+
|desire |0.54 |0.31 |0.40 |83 |0.33 |
|
| 105 |
+
|disappointment|0.31 |0.34 |0.33 |151 |0.22 |
|
| 106 |
+
|disapproval |0.31 |0.57 |0.40 |267 |0.20 |
|
| 107 |
+
|disgust |0.44 |0.40 |0.42 |123 |0.29 |
|
| 108 |
+
|embarrassment |0.48 |0.38 |0.42 |37 |0.18 |
|
| 109 |
+
|excitement |0.29 |0.43 |0.34 |103 |0.29 |
|
| 110 |
+
|fear |0.56 |0.78 |0.65 |78 |0.24 |
|
| 111 |
+
|gratitude |0.95 |0.85 |0.89 |352 |0.71 |
|
| 112 |
+
|grief |0.03 |0.33 |0.05 |6 |0.02 |
|
| 113 |
+
|joy |0.48 |0.58 |0.53 |161 |0.31 |
|
| 114 |
+
|love |0.73 |0.84 |0.78 |238 |0.45 |
|
| 115 |
+
|nervousness |0.24 |0.48 |0.32 |23 |0.06 |
|
| 116 |
+
|optimism |0.57 |0.54 |0.56 |186 |0.18 |
|
| 117 |
+
|pride |0.67 |0.38 |0.48 |16 |0.04 |
|
| 118 |
+
|realization |0.18 |0.31 |0.23 |145 |0.08 |
|
| 119 |
+
|relief |0.30 |0.27 |0.29 |11 |0.10 |
|
| 120 |
+
|remorse |0.53 |0.84 |0.65 |56 |0.22 |
|
| 121 |
+
|sadness |0.56 |0.53 |0.55 |156 |0.39 |
|
| 122 |
+
|surprise |0.55 |0.57 |0.56 |141 |0.35 |
|
| 123 |
+
|neutral |0.59 |0.79 |0.68 |1787 |0.24 |
|
| 124 |
+
|micro avg |0.50 |0.66 |0.57 |6329 | |
|
| 125 |
+
|macro avg |0.46 |0.55 |0.48 |6329 | |
|
| 126 |
+
|weighted avg |0.53 |0.66 |0.58 |6329 | |
|
| 127 |
+
|