Not loading without tokenizer.json
#1
by
Lockout
- opened
raise FileNotFoundError("No supported tokenizer found.")
FileNotFoundError: No supported tokenizer found.
Not sure what to do. Figure out how to convert tokenizer.model, take another tokenizer.json from somewhere?
I'm trying this one: https://huggingface.co/mistral-community/Pixtral-Large-Instruct-2411/blob/main/tokenizer.json
will see what happens
Lockout
changed discussion title from
Not loading with tokenizer.json
to Not loading without tokenizer.json
Ok with that tokenizer I get gibberish and crashes.
it finally works for text at least.