Update README.md
Browse files
README.md
CHANGED
|
@@ -15,42 +15,7 @@ datasets:
|
|
| 15 |
metrics:
|
| 16 |
- rouge
|
| 17 |
inference: false
|
| 18 |
-
|
| 19 |
-
- name: pszemraj/long-t5-tglobal-xl-16384-book-summary
|
| 20 |
-
results:
|
| 21 |
-
- task:
|
| 22 |
-
type: summarization
|
| 23 |
-
name: Summarization
|
| 24 |
-
dataset:
|
| 25 |
-
name: kmfoda/booksum
|
| 26 |
-
type: kmfoda/booksum
|
| 27 |
-
config: kmfoda--booksum
|
| 28 |
-
split: test
|
| 29 |
-
metrics:
|
| 30 |
-
- name: ROUGE-1
|
| 31 |
-
type: rouge
|
| 32 |
-
value: 34.0139
|
| 33 |
-
verified: true
|
| 34 |
-
- name: ROUGE-2
|
| 35 |
-
type: rouge
|
| 36 |
-
value: 7.0901
|
| 37 |
-
verified: true
|
| 38 |
-
- name: ROUGE-L
|
| 39 |
-
type: rouge
|
| 40 |
-
value: 17.0925
|
| 41 |
-
verified: true
|
| 42 |
-
- name: ROUGE-LSUM
|
| 43 |
-
type: rouge
|
| 44 |
-
value: 31.6899
|
| 45 |
-
verified: true
|
| 46 |
-
- name: loss
|
| 47 |
-
type: loss
|
| 48 |
-
value: 2.096665382385254
|
| 49 |
-
verified: true
|
| 50 |
-
- name: gen_len
|
| 51 |
-
type: gen_len
|
| 52 |
-
value: 377.109
|
| 53 |
-
verified: true
|
| 54 |
---
|
| 55 |
|
| 56 |
# long-t5-tglobal-xl + BookSum
|
|
@@ -136,6 +101,21 @@ Official results with the [model evaluator](https://huggingface.co/spaces/autoev
|
|
| 136 |
- eval_samples_per_second: 0.107
|
| 137 |
- eval_steps_per_second: 0.027
|
| 138 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
---
|
| 140 |
|
| 141 |
## FAQ
|
|
|
|
| 15 |
metrics:
|
| 16 |
- rouge
|
| 17 |
inference: false
|
| 18 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
---
|
| 20 |
|
| 21 |
# long-t5-tglobal-xl + BookSum
|
|
|
|
| 101 |
- eval_samples_per_second: 0.107
|
| 102 |
- eval_steps_per_second: 0.027
|
| 103 |
|
| 104 |
+
```
|
| 105 |
+
***** predict/test metrics (initial) *****
|
| 106 |
+
predict_gen_len = 506.4368
|
| 107 |
+
predict_loss = 2.028
|
| 108 |
+
predict_rouge1 = 36.8815
|
| 109 |
+
predict_rouge2 = 8.0625
|
| 110 |
+
predict_rougeL = 17.6161
|
| 111 |
+
predict_rougeLsum = 34.9068
|
| 112 |
+
predict_runtime = 2:04:14.37
|
| 113 |
+
predict_samples = 1431
|
| 114 |
+
predict_samples_per_second = 0.192
|
| 115 |
+
predict_steps_per_second = 0.048
|
| 116 |
+
```
|
| 117 |
+
\* evaluating big model not as easy as it seems. Doing a bit more investigating
|
| 118 |
+
|
| 119 |
---
|
| 120 |
|
| 121 |
## FAQ
|