how to inference this model?
#1
by
xiximayou
- opened
describe as question. Can this model be inferenced by vllm or sglang?
For vllm, you could use this one https://huggingface.co/OPEA/DeepSeek-R1-int4-AutoRound-awq-asym