Is there a way to run it on vLLM?

by Jongsim - opened 22 days ago

22 days ago

Which version of vLLM can run this BnB model?
I want to run the API server using a command.
Since this model is for fine-tuning, is it not possible to run it with vLLM?

Jongsim

22 days ago

•

edited 22 days ago

No description provided.

Jongsim changed discussion status to closed 22 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment