File size: 636 Bytes
87c85b5 a9fee4d 87c85b5 a9fee4d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
---
language: dv
---
# byt5-base-dv
Pretrained from scratch on Dhivei (language of the Maldives)
with ByT5, Google's new byte-level tokenizer strategy.
**Use byt5-dv for now; this is less accurate**
Corpus: Sofwath's Dhivehi corpus https://github.com/Sofwath/DhivehiDatasets
Pretraining Notebook:
https://colab.research.google.com/drive/1ERIZ1PyHn-yN_jo7dTQeODn22vrt-d1d?usp=sharing
## Fine-tuning Demo
On Dhivehi news classification task
https://colab.research.google.com/drive/11u5SafR4bKICmArgDl6KQ9vqfYtDpyWp?usp=sharing
## Issues
There was an issue with the vocabulary size, final layer, and/or accuracy on fine-tuning.
|