sglang和vllm运行目前不支持blockwise_int8

#12

by zhangneilc - opened Mar 3

Mar 3

sglang和vllm运行目前不支持blockwise_int8，这个怎么解

pkumc

meituan org Mar 3

pkumc changed discussion status to closed 26 days ago

WYBJ

5 days ago

how about vllm, have any plan?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment