Extend eos_token_id
#35
by
Wovchena
- opened
Extend eos_token_id
with pad token because that's the token stored in processor.tokenizer.eos_token_id
. That also simplifies the sample from the model card from
generate_ids = model.generate(**inputs,
eos_token_id=processor.tokenizer.eos_token_id,
**generation_args
)
to
generate_ids = model.generate(**inputs,
**generation_args
)
Wovchena
changed pull request title from
Update generation_config.json
to Extend eos_token_id