airesearch

/

wangchanberta-base-att-spm-uncased

like 43

Follow

VISTEC-depa AI Research Institute of Thailand 72

I'd like to run the model with Elixir Bumblebee and it requires this tokenizer config

Generated with:

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("airesearch/wangchanberta-base-att-spm-uncased")
assert tokenizer.is_fast
tokenizer.save_pretrained("...")