请问有对比过w8a8量化后跟bf16解码速度嘛？

by takagi97 - opened Apr 1

Apr 1

我直接解码体验感觉量化后的速度跟bf16差不多？bsz相同的情况下？不知道dalao有没有经验？@yuqian-0513

Apr 1

bf16和w8a8的mindie的配置文件上的区别是不是只是模型路径不一样呀？

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment