how to use this with ollama
#26
by
Pawankumar9413
- opened
I am trying to run this with the ollama but not getting the ouput in proper format i am getting output in this form
does anyone know the solution of this
search for quantized format;
https://huggingface.co/mradermacher/gpt-neox-20b-GGUF
then find size that fits into your RAM comfortably (on the right of the model page) then click on use this model and select Ollama if you have the app selected as an app you use on your HF account and it will provide you with the CLI for Ollama to dowload/run;ollama run hf.co/mradermacher/gpt-neox-20b-GGUF:Q8_0