my-tokenizer / requirements.txt
Max1798's picture
Create requirements.txt
29e1ab5 verified
raw
history blame contribute delete
32 Bytes
from tokenizers import Tokenizer