Hi, how can I convert this to FP16 (to run with vLLM Apple Silicon)?
Hi @sd17js2 ,
Welcome to Gemma models, thanks for reaching out to us. If you would like to explore and experiment with the data types you could do that by utilizing the torch_dtype parameter. To know more about the Gemma 3n models please visit the following page.
torch_dtype
Thanks.
https://huggingface.co/mlx-community/models?search=gemma-3n
· Sign up or log in to comment