Recommended inference settings
What are the recommended inference settings for this model.
on AI studio it shows
Temperature of 1.0
Top_P of 0.95
but on Vertex AI it shows
Temperature of 1.0
Top_P of 1.o
Top_K 1.0
So which one of these it is really?
Hi @hfuserash ,
The optimal inference settings for google/gemma-3-12b-it generally aim to balance creativity with coherence.
Based on official recommendations from the Gemma team (as seen in documentation and community discussions like Unsloth), the AI Studio settings are typically the recommended ones for general use:
- Temperature: 1.0
- Top_P: 0.95
This combination encourages the model to generate diverse and creative responses (due to the higher Temperature) while still ensuring the output remains sensible by focusing on the most probable words.
If your goal is to get creative, varied, and coherent responses, stick with the settings you see in AI Studio: Temperature 1.0 and Top_P 0.95. If you have the option to set Top_K, a value like 64 is commonly recommended in conjunction with Top_P 0.95, but Top_K 1.0 should generally be avoided for this type of model.
Kindly follow this link for more information. Thank you.