Just high-bpw quantization of functionary for a drop-in OpenAI function calling replacement. See the llama-cpp-python docs:

https://llama-cpp-python.readthedocs.io/en/latest/server/

Downloads last month
1
GGUF
Model size
6.74B params
Architecture
llama
Hardware compatibility
Log In to view the estimation

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support