Changes copied from jamesliu1/sglang-EAGLE3-Llama-3.1-Instruct-8B. But also switched torch_dtype from fp16 -> bf16.
This model is copied from https://huggingface.co/yuhuili/EAGLE3-LLaMA3.3-Instruct-70B with the following modifications to make it compatible with SGLang:
Modify the architecture in config.json to LlamaForCausalLMEagle3
- Downloads last month
- 67
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support