[tokenizer] AlternateTokenizer - a minpeter Collection

minpeter 's Collections

[Study] Tiny-ko

[Study] NN MNIST

[Model] FLUX.1 Full Finetuned & Merged

[🛠️] Huggingface Utility

[Dataset] unified standard function calling

[tokenizer] AlternateTokenizer

[Dataset] Function Calling

[tokenizer] AlternateTokenizer

updated Mar 15

A series of modified tokenizers for tuning chatml inspired by teknium and hermes.

minpeter/Llama-3.x-AlternateTokenizer

Updated Feb 11 • 1

Note An alternative tokenizer for the final form of the Llama 3.x series.
teknium/Llama-3.1-AlternateTokenizer

Text Generation • Updated Jul 27, 2024 • 47 • 4
minpeter/Llama-3.2-1B-AlternateTokenizer-chatml

Text Generation • Updated Feb 9 • 21
minpeter/Llama-3.2-1B-AlternateTokenizer-tool-chatml

Text Generation • Updated Feb 10 • 13
philschmid/gemma-tokenizer-chatml

Updated Feb 29, 2024 • 30