I want to download BF16 weight Models, but there's not ready yet.
I want to download BF16 weight Models, but there's not ready yet.
That sucks...
Don't they already train some of the weights using fp8? If so, will bf16 really be better?
Don't they already train some of the weights using fp8? If so, will bf16 really be better?
I guess using bf16 is not for better performance, just because of hardware limitation, not everyone has fp8 devices.
Don't they already train some of the weights using fp8? If so, will bf16 really be better?
I guess using bf16 is not for better performance, just because of hardware limitation, not everyone has fp8 devices.
In theory you can download fp8 and force it to upcast to bf16 when inferencing, not sure if vllm / sglang support this though. Like I can download a bf16 model and use fp32 to do inference.
The originals are FP8, same as the OG R1
This guy has uploaded them upcast to bf16: bf16bullerwins/DeepSeek-R1-0528-bf16