model update

README.md

@@ -2,7 +2,7 @@
 datasets:
 - relbert/semeval2012_relational_similarity
 model-index:
-- name: relbert/relbert-roberta-large-semeval2012-mask-prompt-a-loob
+- name: relbert/roberta-large-semeval2012-mask-prompt-a-loob
   results:
   - task:
       name: Relation Mapping
@@ -14,7 +14,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: None
+      value: 0.9060317460317461
   - task:
       name: Analogy Questions (SAT full)
       type: multiple-choice-qa
@@ -25,7 +25,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: None
+      value: 0.6550802139037433
   - task:
       name: Analogy Questions (SAT)
       type: multiple-choice-qa
@@ -36,7 +36,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: None
+      value: 0.655786350148368
   - task:
       name: Analogy Questions (BATS)
       type: multiple-choice-qa
@@ -47,7 +47,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: None
+      value: 0.8043357420789328
   - task:
       name: Analogy Questions (Google)
       type: multiple-choice-qa
@@ -58,7 +58,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: None
+      value: 0.95
   - task:
       name: Analogy Questions (U2)
       type: multiple-choice-qa
@@ -69,7 +69,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: None
+      value: 0.631578947368421
   - task:
       name: Analogy Questions (U4)
       type: multiple-choice-qa
@@ -80,7 +80,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: None
+      value: 0.6412037037037037
   - task:
       name: Lexical Relation Classification (BLESS)
       type: classification
@@ -91,10 +91,10 @@ model-index:
     metrics:
     - name: F1
       type: f1
-      value: None
+      value: 0.9245140876902215
     - name: F1 (macro)
       type: f1_macro
-      value: None
+      value: 0.9208294548760101
   - task:
       name: Lexical Relation Classification (CogALexV)
       type: classification
@@ -105,10 +105,10 @@ model-index:
     metrics:
     - name: F1
       type: f1
-      value: None
+      value: 0.8814553990610329
     - name: F1 (macro)
       type: f1_macro
-      value: None
+      value: 0.7355497663400952
   - task:
       name: Lexical Relation Classification (EVALution)
       type: classification
@@ -119,10 +119,10 @@ model-index:
     metrics:
     - name: F1
       type: f1
-      value: None
+      value: 0.7128927410617552
     - name: F1 (macro)
       type: f1_macro
-      value: None
+      value: 0.7065924774146382
   - task:
       name: Lexical Relation Classification (K&H+N)
       type: classification
@@ -133,10 +133,10 @@ model-index:
     metrics:
     - name: F1
       type: f1
-      value: None
+      value: 0.9646657856298254
     - name: F1 (macro)
       type: f1_macro
-      value: None
+      value: 0.8945677578632619
   - task:
       name: Lexical Relation Classification (ROOT09)
       type: classification
@@ -147,33 +147,33 @@ model-index:
     metrics:
     - name: F1
       type: f1
-      value: None
+      value: 0.9081792541523034
     - name: F1 (macro)
       type: f1_macro
-      value: None
+      value: 0.906414518159255
 
 ---
-# relbert/relbert-roberta-large-semeval2012-mask-prompt-a-loob
+# relbert/roberta-large-semeval2012-mask-prompt-a-loob
 
 RelBERT fine-tuned from [roberta-large](https://huggingface.co/roberta-large) on  
 [relbert/semeval2012_relational_similarity](https://huggingface.co/datasets/relbert/semeval2012_relational_similarity).
 Fine-tuning is done via [RelBERT](https://github.com/asahi417/relbert) library (see the repository for more detail).
 It achieves the following results on the relation understanding tasks:
-- Analogy Question ([dataset](https://huggingface.co/datasets/relbert/analogy_questions), [full result](https://huggingface.co/relbert/relbert-roberta-large-semeval2012-mask-prompt-a-loob/raw/main/analogy.json)):
-    - Accuracy on SAT (full): None 
-    - Accuracy on SAT: None
-    - Accuracy on BATS: None
-    - Accuracy on U2: None
-    - Accuracy on U4: None
-    - Accuracy on Google: None
-- Lexical Relation Classification ([dataset](https://huggingface.co/datasets/relbert/lexical_relation_classification), [full result](https://huggingface.co/relbert/relbert-roberta-large-semeval2012-mask-prompt-a-loob/raw/main/classification.json)):
-    - Micro F1 score on BLESS: None
-    - Micro F1 score on CogALexV: None
-    - Micro F1 score on EVALution: None
-    - Micro F1 score on K&H+N: None
-    - Micro F1 score on ROOT09: None
-- Relation Mapping ([dataset](https://huggingface.co/datasets/relbert/relation_mapping), [full result](https://huggingface.co/relbert/relbert-roberta-large-semeval2012-mask-prompt-a-loob/raw/main/relation_mapping.json)):
-    - Accuracy on Relation Mapping: None 
+- Analogy Question ([dataset](https://huggingface.co/datasets/relbert/analogy_questions), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-loob/raw/main/analogy.json)):
+    - Accuracy on SAT (full): 0.6550802139037433 
+    - Accuracy on SAT: 0.655786350148368
+    - Accuracy on BATS: 0.8043357420789328
+    - Accuracy on U2: 0.631578947368421
+    - Accuracy on U4: 0.6412037037037037
+    - Accuracy on Google: 0.95
+- Lexical Relation Classification ([dataset](https://huggingface.co/datasets/relbert/lexical_relation_classification), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-loob/raw/main/classification.json)):
+    - Micro F1 score on BLESS: 0.9245140876902215
+    - Micro F1 score on CogALexV: 0.8814553990610329
+    - Micro F1 score on EVALution: 0.7128927410617552
+    - Micro F1 score on K&H+N: 0.9646657856298254
+    - Micro F1 score on ROOT09: 0.9081792541523034
+- Relation Mapping ([dataset](https://huggingface.co/datasets/relbert/relation_mapping), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-loob/raw/main/relation_mapping.json)):
+    - Accuracy on Relation Mapping: 0.9060317460317461 
 
 
 ### Usage
@@ -184,7 +184,7 @@ pip install relbert
 and activate model as below.
 ```python
 from relbert import RelBERT
-model = RelBERT("relbert/relbert-roberta-large-semeval2012-mask-prompt-a-loob")
+model = RelBERT("relbert/roberta-large-semeval2012-mask-prompt-a-loob")
 vector = model.get_embedding(['Tokyo', 'Japan'])  # shape of (1024, )
 ```
 
@@ -211,7 +211,7 @@ The following hyperparameters were used during training:
  - n_sample: 640
  - gradient_accumulation: 8
 
-The full configuration can be found at [fine-tuning parameter file](https://huggingface.co/relbert/relbert-roberta-large-semeval2012-mask-prompt-a-loob/raw/main/trainer_config.json).
+The full configuration can be found at [fine-tuning parameter file](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-loob/raw/main/trainer_config.json).
 
 ### Reference
 If you use any resource from RelBERT, please consider to cite our [paper](https://aclanthology.org/2021.eacl-demos.7/).