ValueError: `rope_scaling` must be a dictionary with with two fields
When deploying my fine tuned model on a dedicated inference endpoint on hugging face this gets triggered:
ValueError: rope_scaling must be a dictionary with with two fields, type and factor, got {'factor': 8.0, 'low_freq_factor': 1.0, 'high_freq_factor': 4.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}
Application startup failed. Exiting.
I can't access the config.json file since its on the base model so no way to modify these value or to upgrade transformers (since its deploying on huggingface servers).
Any idea what to do?
Same issue!
Same here
Same here. Tried llama 3.1 and it does not work.
same here
pip install transformers -U
It works for me.
@yoo how can you run that on a dedicated inference endpoint when the endpoint can not be deployed?
Does anyone found a solution? I am trying to deploy using text generation inference.