Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
Corianas
/
char128_shift_tokenizer
like
0
English
License:
mit
Model card
Files
Files and versions
xet
Community
main
char128_shift_tokenizer
Ctrl+K
Ctrl+K
1 contributor
History:
5 commits
Corianas
Update README.md
4281937
verified
18 days ago
src
Upload build_shift_char_tokenizer.py
18 days ago
.gitattributes
Safe
1.52 kB
initial commit
18 days ago
README.md
4.06 kB
Update README.md
18 days ago
special_tokens_map.json
Safe
105 Bytes
Upload 3 files
18 days ago
tokenizer.json
7.01 kB
Upload 3 files
18 days ago
tokenizer_config.json
1.01 kB
Upload 3 files
18 days ago
