Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
amirhofo
/
Persian-BPE-Tokenizer
like
0
Persian
persian
bpe
tokenizer
License:
mit
Model card
Files
Files and versions
xet
Community
main
Persian-BPE-Tokenizer
2.34 MB
1 contributor
History:
4 commits
amirhofo
Update README.md
eb6970d
verified
3 months ago
.gitattributes
Safe
1.52 kB
initial commit
3 months ago
LICENSE
1.1 kB
Upload files
3 months ago
Persian_BPE_Tokenizer_30K.json
2.32 MB
Upload files
3 months ago
README.md
1.29 kB
Update README.md
3 months ago
main.py
1.78 kB
Upload files
3 months ago
packages.py
201 Bytes
Upload files
3 months ago
preprocess.py
3.31 kB
Upload files
3 months ago
requirements.txt
85 Bytes
Upload files
3 months ago
tokenizer_training.py
1.02 kB
Upload files
3 months ago
utils.py
2.89 kB
Upload files
3 months ago
