Warning message for right side padding even after setting padding_side="left"
#198
by
mbismay
- opened
Has anyone faced the following issue while batch decoding -
I am using Mistral 8x7B v0.1 with 4bit quantization - I have initialized the tokenizer astokenizer = AutoTokenizer.from_pretrained(model_id, use_fast = False, padding_side = "left", add_eos_token = True, add_bos_token = True)
tokenizer.pad_token = tokenizer.eos_token
But, I still get the following warning message -
A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left'
when initializing the tokenizer.