radinplaid commited on
Commit
d8d6087
·
verified ·
1 Parent(s): 3c69df2

Update README.md

Browse files

Remove mention of MADLAD - did not bother benchmarking because it would not fit in GPU RAM on an RTX 4070s with 12 GB memory

Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -74,7 +74,7 @@ The model is in `ctranslate2` format, and the tokenizers are `sentencepiece`, so
74
 
75
  ## Metrics
76
 
77
- BLEU and CHRF2 calculated with [sacrebleu](https://github.com/mjpost/sacrebleu) on the [Flores200 `devtest` test set](https://huggingface.co/datasets/facebook/flores) ("zho_Hans"->"eng_Latn"). COMET22 with the [`comet`](https://github.com/Unbabel/COMET) library and the [default model](https://huggingface.co/Unbabel/wmt22-comet-da). "Time (s)" is the time in seconds to translate (using `ctranslate2`) the flores-devtest dataset (1012 sentences) on an RTX 4070s GPU with batch size 32 except for `madlad400-3b-mt` which used a batch size of 1.
78
 
79
  | Model | bleu | chrf2 | comet22 | Time (s) |
80
  | -------------------------------- | ----- | ----- | ---- | ---- |
 
74
 
75
  ## Metrics
76
 
77
+ BLEU and CHRF2 calculated with [sacrebleu](https://github.com/mjpost/sacrebleu) on the [Flores200 `devtest` test set](https://huggingface.co/datasets/facebook/flores) ("zho_Hans"->"eng_Latn"). COMET22 with the [`comet`](https://github.com/Unbabel/COMET) library and the [default model](https://huggingface.co/Unbabel/wmt22-comet-da). "Time (s)" is the time in seconds to translate (using `ctranslate2`) the flores-devtest dataset (1012 sentences) on an RTX 4070s GPU with batch size 32.
78
 
79
  | Model | bleu | chrf2 | comet22 | Time (s) |
80
  | -------------------------------- | ----- | ----- | ---- | ---- |