OpenAI compatible endpoint using VLLM

Runs well on 4090

python -m vllm.entrypoints.openai.api_server --model macadeliccc/laser-dolphin-mixtral-4x7b-dpo-AWQ  --max-model-len 25000

Safetensors

Model size

3.46B params

Tensor type

I32

FP16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for macadeliccc/laser-dolphin-mixtral-4x7b-dpo-AWQ

Base model

Quantized

(4)

this model