fyaronskiy
/

ruRoberta-large-ru-go-emotions

@@ -1,199 +1,127 @@
 ---
 library_name: transformers
-tags: []
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
 library_name: transformers
+license: mit
+datasets:
+- seara/ru_go_emotions
+language:
+- ru
+metrics:
+- f1
 ---
+This is [ruRoberta-large](https://huggingface.co/ai-forever/ruRoberta-large) model finetuned on [ru_go_emotions](https://huggingface.co/datasets/seara/ru_go_emotions)
+dataset for multilabel classification. Model can be used to extract all emotions from text or detect certain emotions.
+# Usage
+Using model with Huggingface Transformers:
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+tokenizer = AutoTokenizer.from_pretrained("fyaronskiy/ruRoberta-large-ru-go-emotions")
+model = AutoModelForSequenceClassification.from_pretrained("fyaronskiy/ruRoberta-large-ru-go-emotions")
+best_thresholds = [0.36734693877551017, 0.2857142857142857, 0.2857142857142857, 0.16326530612244897, 0.14285714285714285, 0.14285714285714285, 0.18367346938775508, 0.3469387755102041, 0.32653061224489793, 0.22448979591836732, 0.2040816326530612, 0.2857142857142857, 0.18367346938775508, 0.2857142857142857, 0.24489795918367346, 0.7142857142857142, 0.02040816326530612, 0.3061224489795918, 0.44897959183673464, 0.061224489795918366, 0.18367346938775508, 0.04081632653061224, 0.08163265306122448, 0.1020408163265306, 0.22448979591836732, 0.3877551020408163, 0.3469387755102041, 0.24489795918367346]
+LABELS = ['admiration', 'amusement', 'anger', 'annoyance', 'approval', 'caring', 'confusion', 'curiosity', 'desire', 'disappointment', 'disapproval', 'disgust', 'embarrassment', 'excitement', 'fear', 'gratitude', 'grief', 'joy', 'love', 'nervousness', 'optimism', 'pride', 'realization', 'relief', 'remorse', 'sadness', 'surprise', 'neutral']
+ID2LABEL = dict(enumerate(LABELS))
+```
+Here is how you can extract emotions contained in text:
+```python
+def predict_emotions(text):
+  inputs = tokenizer(text, truncation=True, add_special_tokens=True, max_length=128, return_tensors='pt')
+  with torch.no_grad():
+      logits = model(**inputs).logits
+  probas = torch.sigmoid(logits).squeeze(dim=0)
+  probas = probas.cpu().numpy()
+  class_binary_labels = (probas > np.array(best_thresholds)).astype(int)
+  return [ID2LABEL[label_id] for label_id, value in enumerate(class_binary_labels) if value == 1]
+print(predict_emotions('У вас отличный сервис и лучший кофе в городе, обожаю вашу кофейню!'))
+#['admiration', 'love']
+```
+This is the way to get all emotions and their scores:
+```python
+def predict(text):
+    inputs = tokenizer(text, truncation=True, add_special_tokens=True, max_length=128, return_tensors='pt')
+    with torch.no_grad():
+        logits = model(**inputs).logits
+    probas = torch.sigmoid(logits).squeeze(dim=0).tolist()
+    probas = [round(proba, 3) for proba in probas]
+    labels2probas = dict(zip(LABELS, probas))
+    probas_dict_sorted = dict(sorted(labels2probas.items(), key=lambda x: x[1], reverse=True))
+    return probas_dict_sorted
+print(predict('У вас отличный сервис и лучший кофе в городе, обожаю вашу кофейню!'))
+'''{'admiration': 0.81,
+ 'love': 0.538,
+ 'joy': 0.041,
+ 'gratitude': 0.031,
+ 'approval': 0.026,
+ 'excitement': 0.023,
+ 'neutral': 0.009,
+ 'curiosity': 0.006,
+ 'amusement': 0.005,
+ 'desire': 0.005,
+ 'realization': 0.005,
+ 'caring': 0.004,
+ 'confusion': 0.004,
+ 'surprise': 0.004,
+ 'disappointment': 0.003,
+ 'disapproval': 0.003,
+ 'anger': 0.002,
+ 'annoyance': 0.002,
+ 'disgust': 0.002,
+ 'fear': 0.002,
+ 'grief': 0.002,
+ 'optimism': 0.002,
+ 'pride': 0.002,
+ 'relief': 0.002,
+ 'sadness': 0.002,
+ 'embarrassment': 0.001,
+ 'nervousness': 0.001,
+ 'remorse': 0.001}
+'''
+```
+# Eval results on test split of ru-go-emotions
+                precision  recall  f1-score  support threshold
+admiration           0.63    0.75      0.69    504       0.37
+amusement            0.76    0.91      0.83    264       0.29
+anger                0.47    0.32      0.38    198       0.29
+annoyance            0.33    0.39      0.36    320       0.16
+approval             0.27    0.58      0.37    351       0.14
+caring               0.32    0.59      0.41    135       0.14
+confusion            0.41    0.52      0.46    153       0.18
+curiosity            0.45    0.73      0.55    284       0.35
+desire               0.54    0.31      0.40     83       0.33
+disappointment       0.31    0.34      0.33    151       0.22
+disapproval          0.31    0.57      0.40    267       0.20
+disgust              0.44    0.40      0.42    123       0.29
+embarrassment        0.48    0.38      0.42     37       0.18
+excitement           0.29    0.43      0.34    103       0.29
+fear                 0.56    0.78      0.65     78       0.24
+gratitude            0.95    0.85      0.89    352       0.71
+grief                0.03    0.33      0.05      6       0.02
+joy                  0.48    0.58      0.53    161       0.31
+love                 0.73    0.84      0.78    238       0.45
+nervousness          0.24    0.48      0.32     23       0.06
+optimism             0.57    0.54      0.56    186       0.18
+pride                0.67    0.38      0.48     16       0.04
+realization          0.18    0.31      0.23    145       0.08
+relief               0.30    0.27      0.29     11       0.10
+remorse              0.53    0.84      0.65     56       0.22
+sadness              0.56    0.53      0.55    156       0.39
+surprise             0.55    0.57      0.56    141       0.35
+neutral              0.59    0.79      0.68   1787       0.24
+micro avg            0.50    0.66      0.57   6329
+macro avg            0.46    0.55      0.48   6329
+weighted avg         0.53    0.66      0.58   6329
+samples avg          0.55    0.68      0.59   6329