Hugging Face's logo
David Soušek's picture
24 40

David Soušek

sousekd
21world's profile picture anikifoss's profile picture coughmedicine's profile picture
·
https://www.leafdot.net

AI & ML interests

Running AI on-prem with agents processing confidential data. Supporting government, private sector, and all kinds of weirdos.

Recent Activity

new activity 17 days ago
ubergarm/Devstral-2-123B-Instruct-2512-GGUF:Going to fit it into RTX PRO 6000!
liked a model 19 days ago
AesSedai/GLM-4.6-Derestricted-GGUF
replied to csabakecskemeti's post 23 days ago
Looking for some help to test an INT8 Deepseek 3.2: SGLang supports Channel wise INT8 quants on CPUs with AMX instructions (Xeon 5 and above AFAIK) https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/ Currently uploading an INT8 version of Deepseek 3.2 Speciale: https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8 I cannot test this I'm on AMD "AssertionError: W8A8Int8LinearMethod on CPU requires that CPU has AMX support" (I assumed it can fall back to some non optimized kernel but seems not) If anyone with the required resources (Intel Xeon 5/6 + ~768-1TB ram) can help to test this that would be awesome. If you have hints how to make this work on AMD Threadripper 7000 Pro series please guide me. Thanks all!
View all activity

Organizations

None yet