Instructions to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0 # Run inference directly in the terminal: llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0 # Run inference directly in the terminal: llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
Use Docker
docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
- Ollama
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF with Ollama:
ollama run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
- Unsloth Desktop
- Pi
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF with Docker Model Runner:
docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
- Lemonade
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
Run and chat with the model
lemonade run user.Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF-Q2_0
List all available models
lemonade list
- Hermes Agent
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF:Q2_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Sakura Qwen3.8 Flash-Next Compact ยท ISTA Q2_0 ยท 512E ยท CP2
47.56 GiB main model ยท 512 experts ยท Windows / Vulkan ยท MTP included
A smaller experimental community edition of ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Q2_0 quant). The two main-model files shrink from 61.86 to 47.56 GiB, saving 14.30 GiB (23.1%), while retaining all 512 routed experts and 16 lookup heads of the unpruned base model. The matching custom runtime and materialized 512E MTP adapter are included separately.
This is the unpruned Qwen base model (NOT Swift, NOT REAP).
Use the supplied runtime and START.cmd. This experimental custom format requires its matching loader; generic GGUF launch instructions shown automatically by Hugging Face do not establish compatibility.
Made and tested on AMD Strix Halo, Radeon 8060S, 64 GB UMA, Windows x64, using Vulkan. Independent community release; not an official Qwen or ISTA-DASLab release.
At a glance
| Item | This release |
|---|---|
| Model family | Qwen3.8-Flash-Next (base) |
| Architecture | qwen4exp, 48 layers |
| Routed experts | 512 experts retained (full, unpruned; 0 pruned); 10 active |
| Lookup heads | All 16 retained |
| Base quant source | ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Q2_0) |
| Main-model download | 47.56 GiB, two GGUF shards |
| MTP adapter | 2.44 GiB additional; included under mtp/ |
| Main model + MTP | 50.00 GiB, before runtime and other small files |
| Measured generation | ~33โ37 token/s with MTP2 / 38.9โ43.8 token/s in short checks |
| Included starter | One slot, 8K context capacity, Q8_0 KV, MTP2, GPU cache disabled |
| Status | Experimental; limited quality and context testing |
Disk size is not running memory. The tested 8K profile with MTP2 and Q8_0 KV is dimensioned for unified memory systems such as Strix Halo 64 GB UMA. A 47.6 GiB file size does not imply that this model fits a 32 GB or 24 GB GPU.
Hardware compatibility
The status below applies to this Compact CP2 v1 model and the supplied binaries. Vulkan support in a GPU driver alone does not establish model or runtime compatibility.
| Hardware / platform | Status | What to expect |
|---|---|---|
| AMD Strix Halo, Radeon 8060S, 64 GB UMA, Windows x64 | Tested | Supplied Vulkan runtime; one slot, 8K context capacity, Q8_0 KV and MTP2. Speed and test measurements are below. |
| Other Strix Halo configurations, including larger UMA capacities | Not tested | Similar hardware is not a separate validation. No measured speed, maximum context or memory guarantee for these configurations. |
| Other AMD Radeon GPUs on Windows | Not tested | The supplied backend is Vulkan. Driver features, host CPU instructions and available GPU/system memory must also match; no hardware-specific profile has been qualified. |
| NVIDIA GPUs on Windows | Not tested | No CUDA build is supplied. Vulkan operation with this custom format has not been validated on NVIDIA hardware. |
| Intel GPUs on Windows | Not tested | No tested Intel Vulkan profile or performance figures are available. |
| GPUs with 24 or 32 GB dedicated VRAM | Default profile not validated | The tested profile requires unified memory with sufficient headroom on UMA. A smaller-VRAM system may require different offloading and context settings; such profiles have not been tested. |
| CPU-only execution | Not benchmarked or qualified | The included launchers select Vulkan0. No CPU-only speed or RAM requirement is claimed. |
| Linux / macOS / Windows on ARM | No matching native binary supplied | This package contains Windows x64 binaries. Their native use on these platforms is not supported by the supplied package. |
CPU, memory and storage
- CPU: the runtime was compiled as a native build on the test machine and tested on its AVX-512-capable x64 CPU. A portable AVX2-only build is not included. An AVX-512 label by itself is not a compatibility test for every other CPU.
- Unified memory: 64 GB UMA is the tested configuration, not a measured minimum for every platform. Dedicated and shared Windows counters both draw from that same physical memory pool on Strix Halo. Leave room for Windows, CPU allocations and other applications.
- Discrete GPUs: separate system RAM does not provide the same behavior or bandwidth as Strix Halo UMA. Neither 48 GB nor a larger VRAM capacity alone guarantees compatibility; the device, driver, CPU and custom runtime all need validation.
- Storage: allow at least the 50.00 GiB model-plus-MTP footprint, plus the runtime and supporting files. An SSD is recommended for the lazily read table. Free disk space does not substitute for sufficient runtime memory.
Provenance and what changed
The model tree points directly to the common Qwen base. Its actual lineage is:
- Qwen/Qwen3.8-Flash-Next.
- ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Q2_0 quantization).
- This Compact CP2 v1 conversion.
Important distinctions:
- No Swift contribution: Unlike the sister release
webmp3/Sakura-Qwen3.8-Flash-Next-Compact-Swift-IQ3_XXS-352E-CP2-GGUF, this release is based directly on the unpruned Qwen base quant. - 512 experts: All 512 routed experts are preserved. No expert pruning (REAP/Swift) was applied.
- Tensor preservation: All expert weights and all non-lookup tensor bytes are 100% byte-identical to the source ISTA Q2_0 model. Only the large per-layer token lookup table was compressed using the CP2 lookup format, and associated GGUF metadata/filenames were updated.
- Tokenizer preservation: Target model's original tokenizer and chat template (
<|im_start|>, Jinja template, etc.) are strictly preserved. - MTP adapter: Materialized for this exact 512-expert model from the shared weights without retraining.
The package also includes a matching native runtime and a materialized MTP adapter for this exact 512E model. See NOTICE.md for the change notice and attribution.
Files
| File or directory | Size / purpose |
|---|---|
Sakura-Qwen3.8-Flash-Next-K512-Q2_0-CP2-v1-Experimental-00001-of-00002.gguf |
35.04 GiB โ load this first shard |
Sakura-Qwen3.8-Flash-Next-K512-Q2_0-CP2-v1-Experimental-00002-of-00002.gguf |
12.52 GiB โ second shard, required alongside the first |
mtp/mtp-Qwen3.8-Flash-Next-K512-materialized-Agention.gguf |
2.44 GiB โ matching MTP adapter for 512 experts |
runtime/ |
Matching Windows x64 Vulkan binaries, DLLs and software license notices |
START.cmd |
Default launcher (calls Start-Sakura-CP2.cmd) |
Start-Sakura-CP2.cmd |
Server launch script configured for 8K context, MTP2, Vulkan |
results.json |
Public measurement summary and evaluation corpus hashes |
SHA256SUMS |
Release file checksums |
LICENSE-QWEN, NOTICE.md |
Model licenses, attribution and changes |
Keep both main shards in the same folder. Select shard 1; the runtime locates shard 2 automatically.
Hugging Face hardware widget: Q2_0 in the filenames identifies the retained source expert-quantization tier; CP2 identifies the custom lookup-table conversion. The widget may group this hybrid release under Q2_0. That label is not a certification of hardware fit or stock-loader compatibility: the included CP2 runtime is still required.
Start on Windows
Download the package with its folder layout intact. Keep the model shards, launchers, mtp/, runtime/ and license files together. No conversion is needed to use the supplied package.
- Double-click START.cmd in the release folder.
- Leave its terminal open while using the model.
- Connect your OpenAI-compatible client to http://127.0.0.1:8080/v1. Query
/v1/modelsfor the model ID to use in the client. - Stop the server with Ctrl+C in its terminal.
The starter serves a local API only (--no-webui); it does not open a browser chat interface. It binds to 127.0.0.1.
Included profile
| Setting | Value |
|---|---|
| Backend | Vulkan, Vulkan0 |
| GPU layers | -ngl 99 |
| Context capacity | 8,192 tokens |
| Concurrent slots | 1 |
| Batch / microbatch | 512 / 512 |
| KV cache | Q8_0 for K and V |
| Flash attention | On |
| Table loading | -lm dio --lazy-mode on |
| Speculation | draft-mtp, maximum 2 draft tokens |
| Optional GPU row cache | 0 (disabled) |
Manual equivalent
Run from the release folder in Windows Command Prompt:
set "SAKURA_CP2_GPU_CACHE_MIB=0"
runtime\llama-server.exe ^
-m Sakura-Qwen3.8-Flash-Next-K512-Q2_0-CP2-v1-Experimental-00001-of-00002.gguf ^
--host 127.0.0.1 --port 8080 --parallel 1 ^
--device Vulkan0 -ngl 99 -c 8192 -b 512 -ub 512 ^
-fa on -ctk q8_0 -ctv q8_0 -lm dio --lazy-mode on --jinja ^
--spec-type draft-mtp ^
--spec-draft-model mtp\mtp-Qwen3.8-Flash-Next-K512-materialized-Agention.gguf ^
--spec-draft-ngl 99 --spec-draft-device Vulkan0 ^
--spec-draft-n-max 2 --spec-draft-n-min 0 --no-webui
Use the provided 512E adapter with this model. Other shared MTP heads are not interchangeable.
Measured quality: original ISTA Q2_0 versus Compact CP2
Own measurements on these exact conversion inputs and outputs. Lower perplexity is better.
| Fixed text slice | Original Q2_0 | Compact CP2 | Relative PPL change |
|---|---|---|---|
| English (8 chunks) | 2.0115 | 2.0054 | -0.30% |
| Code (8 chunks) | 5.1468 | 5.1335 | -0.26% |
| German (8 chunks) | 2.1204 | 2.1421 | +1.02% |
| German (40 chunks) | 2.1289 | 2.1542 | +1.19% |
Each slice used 512-token chunks. Corpus identifiers are recorded in results.json. In English and Code, perplexity remained virtually flat (-0.30% / -0.26%). In German, perplexity rose by +1.02% on 8 chunks and +1.19% on 40 chunks; these deviations are retained and openly documented. Per user priority, testing focused on English and code without further German optimization. These small changes do not establish general quality superiority.
Generation speed and MTP speculation: own tests
Hardware: AMD Radeon 8060S, Strix Halo, 64 GB UMA, Windows, Vulkan. Tests dated 20 October 2026.
- Generation speed: approximately 33โ37 token/s under sustained generation with MTP2; short 256-token smoke tests reached 38.92 / 43.78 token/s (English) and 41.79 token/s (Code).
- MTP speculation acceptance: 64.8โ69.5% with the supplied materialized 512E MTP adapter (
--spec-type draft-mtp --spec-draft-n-max 2). - Answer parity: Greedy short answer checks (e.g. Paris, 42) matched identically with and without MTP.
A broad standardized coding correctness evaluation has not been performed for this conversion. The model is an experimental release; no quality or correctness guarantee is claimed.
Compatibility and known limits
- Matching custom runtime required. CP2 v1 is an experimental custom tensor format, not a stock GGUF quantization preset. Stock llama.cpp, Maya, LM Studio, Ollama and unrelated forks have not been qualified for this package.
- The included native binaries were tested on Windows x64, an AVX-512-capable CPU and Radeon 8060S with Vulkan. Other CPUs, GPUs, Linux and macOS are untested. Do not mix DLLs from unrelated runtime packages.
- The runtime derives from Agention/llama.cpp and ggml-org/llama.cpp. The supplied build adds support for this Compact model. It has not been demonstrated to be the fastest available fork.
- Deep-context correctness above 8K, 128K operation, multiple concurrent slots, tool use and vision are unqualified for this conversion. No vision projector is included.
- Compact retains all 512 experts of the base model and adds lossy lookup-table compression.
- The supplied starter is a tested starting profile on one machine. Model file size alone is not a system-memory or VRAM requirement.
Licenses and credits
Model weights retain the Qwen Community License 1.0. Read the original license text before deployment. The native software's MIT and third-party notices are included under runtime/; software licenses do not replace the model-weight licenses.
Thanks to Qwen, ISTA-DASLab, Agention and the llama.cpp / GGML contributors for the upstream model, quantization and runtime work. Keep the supplied license texts and NOTICE.md when redistributing.
ไธญๆ็ฎ่ฆ่ฏดๆ ยท ๆจฑ่ฑ Compact 512E
่ฟๆฏๅบไบ ISTA-DASLab GSQ-RCO Q2_0 ็ๅ จ 512 ไธๅฎถ็ Sakura ๅฎ้ชๆง็ดงๅ็ๆฌใไธปๆจกๅ็ไธคไธช GGUF ๅ็ไป 61.86 GiB ็ผฉๅฐๅฐ 47.56 GiB๏ผ่็ 14.30 GiB๏ผ23.1%๏ผใไฟ็ๆชๅชๆๅบ็กๆจกๅ็ๅ จ้จ 512 ไธชไธๅฎถๅ 16 ไธชๆฅๆพๅคด๏ผไธๅฎถๆ้ๅๅ ถไป้ๆฅๆพ่กจๅผ ้ๅญ่ 100% ็ธๅใ้ Swift ๆจกๅ๏ผๆ REAP ๅชๆใ ๆฅๆพ่กจ้็จๆๆ CP2 ๅ็ผฉใ
่ฏทไฝฟ็จๆฌไปๅบ้ๅธฆ็่ฟ่กๆถๅ START.cmdใ 512E ไธๅฑ MTP ้้
ๅจๅฆๅ 2.44 GiBใไฟๆ็ฎๅฝ็ปๆๅฎๆด๏ผๅฏๅจๅๅฎขๆท็ซฏ่ฟๆฅ http://127.0.0.1:8080/v1๏ผๅฏๅจ่ๆฌๅชๆไพๆฌๅฐ API๏ผๆฒกๆ็ฝ้กต่ๅคฉ็้ขใ
ๆต่ฏๅนณๅฐ๏ผWindows x64ใๆฏๆ AVX-512 ็ CPUใAMD Strix Halo Radeon 8060Sใ64 GB ็ปไธๅ ๅญใVulkanใๆต่ฏ้ ็ฝฎไธบ 8K ไธไธๆๅฎน้ใๅๆงฝไฝใQ8_0 KVใMTP2๏ผGPU ่ก็ผๅญๅ ณ้ญใๅฎๆตๆ็ปญ็ๆ้ๅบฆ็บฆ 33โ37 token/s๏ผ็ญๆต่ฏไธญ่พพ 38.9โ43.8 token/sใ้ ๅฅ 512E MTP ้้ ๅจ็ๆๆบๆฅๅ็ไธบ **64.8โ69.5%**ใ่ดชๅฉช็ฎ็ญ็ญๆกๅจๅฏ็จ/็ฆ็จ MTP ไธๅฎๅ จไธ่ดใ็ฃ็ๅคงๅฐไธ็ญไบ่ฟ่กๅ ๅญ้ๆฑ๏ผๅ ถไปๅนณๅฐๆช้ช่ฏใ
่ฑ่ฏญๅไปฃ็ ๅฐๆๅบฆๅบๆฌๆๅนณ๏ผ-0.30% / -0.26%๏ผ๏ผๅพท่ฏญๅฐๆๅบฆ็ฅๆไธๅ๏ผ8ๅ +1.02%๏ผ40ๅ +1.19%๏ผใ่ฏญๆๅๅธ่ง results.jsonใๆจกๅๅคไบๅฎ้ช็ถๆ๏ผไธๆไพ้็จ่ดจ้ไฟ่ฏใ
ๅฎๆดๆฅๆบใๆไปถๅ่กจใๆต่ฏๆกไปถๅ้ๅถ่งไธๆน่ฑๆ่ฏดๆใๆจกๅๆ้ๅ้ๅธฆ็ Qwen ่ฎธๅฏ็บฆๆ๏ผ่ฟ่กๆถ่ฝฏไปถ่ฎธๅฏไธ่ฝๆฟไปฃๆจกๅ่ฎธๅฏใ่ฟๆฏ็ฌ็ซ็คพๅบ็ๆฌ๏ผไธๆฏ Qwen ๆ ISTA-DASLab ๅฎๆนๅๅธใ
- Downloads last month
- 516
2-bit
Model tree for webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF
Base model
Qwen/Qwen3.8-Flash-Next