本地部署72b时，模型输出为空，怎么解决？

#21

by Cranegu - opened 2 days ago

2 days ago

•

多卡本地部署72b，模型输出为空，模型处理结果执行到|im_start|>assistant\n时，就停止了，导致最后解码的输出结果为空。

fensz

about 13 hours ago

我也遇到了同样的问题，我部署的是awq版本，请问有人解决了吗

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment