Error on Loading the Tokenizer

#4
by Exqrch - opened

Hi there!
Is there an example code you guys can share?

I tried using

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="indobenchmark/indogpt")

and

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("indobenchmark/indogpt")
model = AutoModelForCausalLM.from_pretrained("indobenchmark/indogpt")

But both of these code leads to the same error:
ValueError: Tokenizer class IndoNLGTokenizer does not exist or is not currently imported.

Sign up or log in to comment