zai-org/GLM-5-FP8
Text Generation • 754B • Updated • 4.37M • 147
Duyet Le
duyet
AI & ML interests
None yet
Recent Activity
liked a model 5 days ago
zai-org/GLM-5-FP8 liked a model 6 days ago
Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled liked a dataset 13 days ago
Crownelius/Opus-4.6-Reasoning-3300xOrganizations
liked a
model 5 days ago
liked a
model 6 days ago
liked a
dataset 13 days ago
reactedto kostakoff's post with 👍 13 days ago
Post
2096
Mining GPU Nvidia CMP 170HX - let's run some models!
To satisfy my curiosity, I investigated different GPUs and found this: a mining version of the A100 — the CMP 170HX.
It is a very interesting GPU. Based on public documentation, it has hardware similar to the datacenter A100. If you open it up and look at the board, you will see that it's very similar to an A100 board; it even has NVLink connectors.
Online, I found almost no information about how to run it, whether it works with LLMs, or if it's supported by default Nvidia drivers and CUDA. So, I decided to test it myself.
I installed it in my lab (see previous post https://huggingface.co/posts/kostakoff/584269728210158) and found that the default nvidia-driver-570 works with it out of the box. After that, I checked if CUDA was available, and it worked too.
The next step was to try running some models:
- Stable Diffusion XL with BNB4 quantization: It took around two minutes to generate an image, but it works!
- Compiled llama.cpp for CUDA (https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#compilation): I run Mistral 7B Q4_K_M, and this actually worked even better. It was able to generate 33 tokens per second and read 400 tokens per second.
There are some limitations related to power utilization:
- When running PyTorch, it doesn't utilize more than 80 watts.
- When running llama.cpp, utilization is a bit better but still limited to 113 watts.
I found this GitHub thread about the Nvidia CMP https://github.com/dartraiden/NVIDIA-patcher/issues/73, and it looks like this mining GPU has an internal rate limiter based on FMA compute calls. I haven't found a solution to bypass it yet.
llmlaba
To satisfy my curiosity, I investigated different GPUs and found this: a mining version of the A100 — the CMP 170HX.
It is a very interesting GPU. Based on public documentation, it has hardware similar to the datacenter A100. If you open it up and look at the board, you will see that it's very similar to an A100 board; it even has NVLink connectors.
Online, I found almost no information about how to run it, whether it works with LLMs, or if it's supported by default Nvidia drivers and CUDA. So, I decided to test it myself.
I installed it in my lab (see previous post https://huggingface.co/posts/kostakoff/584269728210158) and found that the default nvidia-driver-570 works with it out of the box. After that, I checked if CUDA was available, and it worked too.
The next step was to try running some models:
- Stable Diffusion XL with BNB4 quantization: It took around two minutes to generate an image, but it works!
- Compiled llama.cpp for CUDA (https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#compilation): I run Mistral 7B Q4_K_M, and this actually worked even better. It was able to generate 33 tokens per second and read 400 tokens per second.
There are some limitations related to power utilization:
- When running PyTorch, it doesn't utilize more than 80 watts.
- When running llama.cpp, utilization is a bit better but still limited to 113 watts.
I found this GitHub thread about the Nvidia CMP https://github.com/dartraiden/NVIDIA-patcher/issues/73, and it looks like this mining GPU has an internal rate limiter based on FMA compute calls. I haven't found a solution to bypass it yet.
- 1 reply
liked a
model 14 days ago
liked a
model 16 days ago
liked a
model 17 days ago
upvoted a collection about 1 month ago
liked 3
models about 1 month ago
updated a
collection about 1 month ago
liked a
model about 1 month ago
reactedto danielhanchen's post with 🔥 about 1 month ago
Post
3780
Qwen releases Qwen3-Coder-Next! 💜 Run the locally on 46GB RAM or less.
Thhe model excels at agentic coding & local use. With 256K context, it delivers similar performance to models with 10-20× more active parameters.
GGUF: unsloth/Qwen3-Coder-Next-GGUF
Guide: https://unsloth.ai/docs/models/qwen3-coder-next
Thhe model excels at agentic coding & local use. With 256K context, it delivers similar performance to models with 10-20× more active parameters.
GGUF: unsloth/Qwen3-Coder-Next-GGUF
Guide: https://unsloth.ai/docs/models/qwen3-coder-next
- 10 replies
liked a
model about 1 month ago
liked 4
models about 2 months ago
nvidia/parakeet-ctc-0.6b-Vietnamese
Automatic Speech Recognition • Updated • 639 • 80
stepfun-ai/Step3-VL-10B
Image-Text-to-Text • 10B • Updated • 163k • 398
mlx-community/GLM-4.7-Flash-4bit
Text Generation • Updated • 33.3k • 58
unsloth/GLM-4.7-Flash-GGUF
Text Generation • 30B • Updated • 337k • 568