INT4 LLMs for vLLM
Collection
Accurate INT4 quantized models by Neural Magic, ready for use with vLLM!
•
18 items
•
Updated
•
7
Venkatesan Krishnamurthy
venkilfc
·
AI & ML interests
None yet
Recent Activity
new activity
about 1 month ago
google/gemma-3-27b-it-qat-q4_0-gguf:KeyError: 'general.name'
new activity
2 months ago
Qwen/QwQ-32B:When will you fix the model replies missing</think>\n start tags
upvoted
a
collection
8 months ago
INT4 LLMs for vLLM
Organizations
None yet