Can you provide a FP8 version?

#11

by xjpang85 - opened Jan 17

Discussion

xjpang85

Jan 17

Can you provide a FP8 version for less GPUs.

MiniMax-AI

MiniMax org Jan 18

Can the int8 or int4 version meet the requirements?

21world

Jan 18

int4 ok

zzzsssqqq

MiniMax org 1 day ago

We have submitted a 13454 to vllm, and the inference performance improvement compared to the Hugging Face's implementation is very significant. You can give it a try.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment