Sakura Qwen3.8 Flash-Next Compact ยท ISTA Q2_0 ยท 512E ยท CP2

Sakura logo

47.56 GiB main model ยท 512 experts ยท Windows / Vulkan ยท MTP included

A smaller experimental community edition of ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Q2_0 quant). The two main-model files shrink from 61.86 to 47.56 GiB, saving 14.30 GiB (23.1%), while retaining all 512 routed experts and 16 lookup heads of the unpruned base model. The matching custom runtime and materialized 512E MTP adapter are included separately.

This is the unpruned Qwen base model (NOT Swift, NOT REAP).

Use the supplied runtime and START.cmd. This experimental custom format requires its matching loader; generic GGUF launch instructions shown automatically by Hugging Face do not establish compatibility.

Made and tested on AMD Strix Halo, Radeon 8060S, 64 GB UMA, Windows x64, using Vulkan. Independent community release; not an official Qwen or ISTA-DASLab release.

At a glance

Item This release
Model family Qwen3.8-Flash-Next (base)
Architecture qwen4exp, 48 layers
Routed experts 512 experts retained (full, unpruned; 0 pruned); 10 active
Lookup heads All 16 retained
Base quant source ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Q2_0)
Main-model download 47.56 GiB, two GGUF shards
MTP adapter 2.44 GiB additional; included under mtp/
Main model + MTP 50.00 GiB, before runtime and other small files
Measured generation ~33โ€“37 token/s with MTP2 / 38.9โ€“43.8 token/s in short checks
Included starter One slot, 8K context capacity, Q8_0 KV, MTP2, GPU cache disabled
Status Experimental; limited quality and context testing

Disk size is not running memory. The tested 8K profile with MTP2 and Q8_0 KV is dimensioned for unified memory systems such as Strix Halo 64 GB UMA. A 47.6 GiB file size does not imply that this model fits a 32 GB or 24 GB GPU.

Hardware compatibility

The status below applies to this Compact CP2 v1 model and the supplied binaries. Vulkan support in a GPU driver alone does not establish model or runtime compatibility.

Hardware / platform Status What to expect
AMD Strix Halo, Radeon 8060S, 64 GB UMA, Windows x64 Tested Supplied Vulkan runtime; one slot, 8K context capacity, Q8_0 KV and MTP2. Speed and test measurements are below.
Other Strix Halo configurations, including larger UMA capacities Not tested Similar hardware is not a separate validation. No measured speed, maximum context or memory guarantee for these configurations.
Other AMD Radeon GPUs on Windows Not tested The supplied backend is Vulkan. Driver features, host CPU instructions and available GPU/system memory must also match; no hardware-specific profile has been qualified.
NVIDIA GPUs on Windows Not tested No CUDA build is supplied. Vulkan operation with this custom format has not been validated on NVIDIA hardware.
Intel GPUs on Windows Not tested No tested Intel Vulkan profile or performance figures are available.
GPUs with 24 or 32 GB dedicated VRAM Default profile not validated The tested profile requires unified memory with sufficient headroom on UMA. A smaller-VRAM system may require different offloading and context settings; such profiles have not been tested.
CPU-only execution Not benchmarked or qualified The included launchers select Vulkan0. No CPU-only speed or RAM requirement is claimed.
Linux / macOS / Windows on ARM No matching native binary supplied This package contains Windows x64 binaries. Their native use on these platforms is not supported by the supplied package.

CPU, memory and storage

  • CPU: the runtime was compiled as a native build on the test machine and tested on its AVX-512-capable x64 CPU. A portable AVX2-only build is not included. An AVX-512 label by itself is not a compatibility test for every other CPU.
  • Unified memory: 64 GB UMA is the tested configuration, not a measured minimum for every platform. Dedicated and shared Windows counters both draw from that same physical memory pool on Strix Halo. Leave room for Windows, CPU allocations and other applications.
  • Discrete GPUs: separate system RAM does not provide the same behavior or bandwidth as Strix Halo UMA. Neither 48 GB nor a larger VRAM capacity alone guarantees compatibility; the device, driver, CPU and custom runtime all need validation.
  • Storage: allow at least the 50.00 GiB model-plus-MTP footprint, plus the runtime and supporting files. An SSD is recommended for the lazily read table. Free disk space does not substitute for sufficient runtime memory.

Provenance and what changed

The model tree points directly to the common Qwen base. Its actual lineage is:

  1. Qwen/Qwen3.8-Flash-Next.
  2. ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Q2_0 quantization).
  3. This Compact CP2 v1 conversion.

Important distinctions:

  • No Swift contribution: Unlike the sister release webmp3/Sakura-Qwen3.8-Flash-Next-Compact-Swift-IQ3_XXS-352E-CP2-GGUF, this release is based directly on the unpruned Qwen base quant.
  • 512 experts: All 512 routed experts are preserved. No expert pruning (REAP/Swift) was applied.
  • Tensor preservation: All expert weights and all non-lookup tensor bytes are 100% byte-identical to the source ISTA Q2_0 model. Only the large per-layer token lookup table was compressed using the CP2 lookup format, and associated GGUF metadata/filenames were updated.
  • Tokenizer preservation: Target model's original tokenizer and chat template (<|im_start|>, Jinja template, etc.) are strictly preserved.
  • MTP adapter: Materialized for this exact 512-expert model from the shared weights without retraining.

The package also includes a matching native runtime and a materialized MTP adapter for this exact 512E model. See NOTICE.md for the change notice and attribution.

Files

File or directory Size / purpose
Sakura-Qwen3.8-Flash-Next-K512-Q2_0-CP2-v1-Experimental-00001-of-00002.gguf 35.04 GiB โ€” load this first shard
Sakura-Qwen3.8-Flash-Next-K512-Q2_0-CP2-v1-Experimental-00002-of-00002.gguf 12.52 GiB โ€” second shard, required alongside the first
mtp/mtp-Qwen3.8-Flash-Next-K512-materialized-Agention.gguf 2.44 GiB โ€” matching MTP adapter for 512 experts
runtime/ Matching Windows x64 Vulkan binaries, DLLs and software license notices
START.cmd Default launcher (calls Start-Sakura-CP2.cmd)
Start-Sakura-CP2.cmd Server launch script configured for 8K context, MTP2, Vulkan
results.json Public measurement summary and evaluation corpus hashes
SHA256SUMS Release file checksums
LICENSE-QWEN, NOTICE.md Model licenses, attribution and changes

Keep both main shards in the same folder. Select shard 1; the runtime locates shard 2 automatically.

Hugging Face hardware widget: Q2_0 in the filenames identifies the retained source expert-quantization tier; CP2 identifies the custom lookup-table conversion. The widget may group this hybrid release under Q2_0. That label is not a certification of hardware fit or stock-loader compatibility: the included CP2 runtime is still required.

Start on Windows

Download the package with its folder layout intact. Keep the model shards, launchers, mtp/, runtime/ and license files together. No conversion is needed to use the supplied package.

  1. Double-click START.cmd in the release folder.
  2. Leave its terminal open while using the model.
  3. Connect your OpenAI-compatible client to http://127.0.0.1:8080/v1. Query /v1/models for the model ID to use in the client.
  4. Stop the server with Ctrl+C in its terminal.

The starter serves a local API only (--no-webui); it does not open a browser chat interface. It binds to 127.0.0.1.

Included profile

Setting Value
Backend Vulkan, Vulkan0
GPU layers -ngl 99
Context capacity 8,192 tokens
Concurrent slots 1
Batch / microbatch 512 / 512
KV cache Q8_0 for K and V
Flash attention On
Table loading -lm dio --lazy-mode on
Speculation draft-mtp, maximum 2 draft tokens
Optional GPU row cache 0 (disabled)

Manual equivalent

Run from the release folder in Windows Command Prompt:

set "SAKURA_CP2_GPU_CACHE_MIB=0"
runtime\llama-server.exe ^
  -m Sakura-Qwen3.8-Flash-Next-K512-Q2_0-CP2-v1-Experimental-00001-of-00002.gguf ^
  --host 127.0.0.1 --port 8080 --parallel 1 ^
  --device Vulkan0 -ngl 99 -c 8192 -b 512 -ub 512 ^
  -fa on -ctk q8_0 -ctv q8_0 -lm dio --lazy-mode on --jinja ^
  --spec-type draft-mtp ^
  --spec-draft-model mtp\mtp-Qwen3.8-Flash-Next-K512-materialized-Agention.gguf ^
  --spec-draft-ngl 99 --spec-draft-device Vulkan0 ^
  --spec-draft-n-max 2 --spec-draft-n-min 0 --no-webui

Use the provided 512E adapter with this model. Other shared MTP heads are not interchangeable.

Measured quality: original ISTA Q2_0 versus Compact CP2

Own measurements on these exact conversion inputs and outputs. Lower perplexity is better.

Fixed text slice Original Q2_0 Compact CP2 Relative PPL change
English (8 chunks) 2.0115 2.0054 -0.30%
Code (8 chunks) 5.1468 5.1335 -0.26%
German (8 chunks) 2.1204 2.1421 +1.02%
German (40 chunks) 2.1289 2.1542 +1.19%

Each slice used 512-token chunks. Corpus identifiers are recorded in results.json. In English and Code, perplexity remained virtually flat (-0.30% / -0.26%). In German, perplexity rose by +1.02% on 8 chunks and +1.19% on 40 chunks; these deviations are retained and openly documented. Per user priority, testing focused on English and code without further German optimization. These small changes do not establish general quality superiority.

Generation speed and MTP speculation: own tests

Hardware: AMD Radeon 8060S, Strix Halo, 64 GB UMA, Windows, Vulkan. Tests dated 20 October 2026.

  • Generation speed: approximately 33โ€“37 token/s under sustained generation with MTP2; short 256-token smoke tests reached 38.92 / 43.78 token/s (English) and 41.79 token/s (Code).
  • MTP speculation acceptance: 64.8โ€“69.5% with the supplied materialized 512E MTP adapter (--spec-type draft-mtp --spec-draft-n-max 2).
  • Answer parity: Greedy short answer checks (e.g. Paris, 42) matched identically with and without MTP.

A broad standardized coding correctness evaluation has not been performed for this conversion. The model is an experimental release; no quality or correctness guarantee is claimed.

Compatibility and known limits

  • Matching custom runtime required. CP2 v1 is an experimental custom tensor format, not a stock GGUF quantization preset. Stock llama.cpp, Maya, LM Studio, Ollama and unrelated forks have not been qualified for this package.
  • The included native binaries were tested on Windows x64, an AVX-512-capable CPU and Radeon 8060S with Vulkan. Other CPUs, GPUs, Linux and macOS are untested. Do not mix DLLs from unrelated runtime packages.
  • The runtime derives from Agention/llama.cpp and ggml-org/llama.cpp. The supplied build adds support for this Compact model. It has not been demonstrated to be the fastest available fork.
  • Deep-context correctness above 8K, 128K operation, multiple concurrent slots, tool use and vision are unqualified for this conversion. No vision projector is included.
  • Compact retains all 512 experts of the base model and adds lossy lookup-table compression.
  • The supplied starter is a tested starting profile on one machine. Model file size alone is not a system-memory or VRAM requirement.

Licenses and credits

Model weights retain the Qwen Community License 1.0. Read the original license text before deployment. The native software's MIT and third-party notices are included under runtime/; software licenses do not replace the model-weight licenses.

Thanks to Qwen, ISTA-DASLab, Agention and the llama.cpp / GGML contributors for the upstream model, quantization and runtime work. Keep the supplied license texts and NOTICE.md when redistributing.


ไธญๆ–‡็ฎ€่ฆ่ฏดๆ˜Ž ยท ๆจฑ่Šฑ Compact 512E

่ฟ™ๆ˜ฏๅŸบไบŽ ISTA-DASLab GSQ-RCO Q2_0 ็š„ๅ…จ 512 ไธ“ๅฎถ็‰ˆ Sakura ๅฎž้ชŒๆ€ง็ดงๅ‡‘็‰ˆๆœฌใ€‚ไธปๆจกๅž‹็š„ไธคไธช GGUF ๅˆ†็‰‡ไปŽ 61.86 GiB ็ผฉๅฐๅˆฐ 47.56 GiB๏ผŒ่Š‚็œ 14.30 GiB๏ผˆ23.1%๏ผ‰ใ€‚ไฟ็•™ๆœชๅ‰ชๆžๅŸบ็ก€ๆจกๅž‹็š„ๅ…จ้ƒจ 512 ไธชไธ“ๅฎถๅ’Œ 16 ไธชๆŸฅๆ‰พๅคด๏ผ›ไธ“ๅฎถๆƒ้‡ๅŠๅ…ถไป–้žๆŸฅๆ‰พ่กจๅผ ้‡ๅญ—่Š‚ 100% ็›ธๅŒใ€‚้ž Swift ๆจกๅž‹๏ผŒๆ—  REAP ๅ‰ชๆžใ€‚ ๆŸฅๆ‰พ่กจ้‡‡็”จๆœ‰ๆŸ CP2 ๅŽ‹็ผฉใ€‚

่ฏทไฝฟ็”จๆœฌไป“ๅบ“้™„ๅธฆ็š„่ฟ่กŒๆ—ถๅ’Œ START.cmdใ€‚ 512E ไธ“ๅฑž MTP ้€‚้…ๅ™จๅฆๅ  2.44 GiBใ€‚ไฟๆŒ็›ฎๅฝ•็ป“ๆž„ๅฎŒๆ•ด๏ผŒๅฏๅŠจๅŽๅฎขๆˆท็ซฏ่ฟžๆŽฅ http://127.0.0.1:8080/v1๏ผ›ๅฏๅŠจ่„šๆœฌๅชๆไพ›ๆœฌๅœฐ API๏ผŒๆฒกๆœ‰็ฝ‘้กต่Šๅคฉ็•Œ้ขใ€‚

ๆต‹่ฏ•ๅนณๅฐ๏ผšWindows x64ใ€ๆ”ฏๆŒ AVX-512 ็š„ CPUใ€AMD Strix Halo Radeon 8060Sใ€64 GB ็ปŸไธ€ๅ†…ๅญ˜ใ€Vulkanใ€‚ๆต‹่ฏ•้…็ฝฎไธบ 8K ไธŠไธ‹ๆ–‡ๅฎน้‡ใ€ๅ•ๆงฝไฝใ€Q8_0 KVใ€MTP2๏ผŒGPU ่กŒ็ผ“ๅญ˜ๅ…ณ้—ญใ€‚ๅฎžๆต‹ๆŒ็ปญ็”Ÿๆˆ้€Ÿๅบฆ็บฆ 33โ€“37 token/s๏ผŒ็Ÿญๆต‹่ฏ•ไธญ่พพ 38.9โ€“43.8 token/sใ€‚้…ๅฅ— 512E MTP ้€‚้…ๅ™จ็š„ๆŠ•ๆœบๆŽฅๅ—็އไธบ **64.8โ€“69.5%**ใ€‚่ดชๅฉช็ฎ€็Ÿญ็ญ”ๆกˆๅœจๅฏ็”จ/็ฆ็”จ MTP ไธ‹ๅฎŒๅ…จไธ€่‡ดใ€‚็ฃ็›˜ๅคงๅฐไธ็ญ‰ไบŽ่ฟ่กŒๅ†…ๅญ˜้œ€ๆฑ‚๏ผ›ๅ…ถไป–ๅนณๅฐๆœช้ชŒ่ฏใ€‚

่‹ฑ่ฏญๅ’Œไปฃ็ ๅ›ฐๆƒ‘ๅบฆๅŸบๆœฌๆŒๅนณ๏ผˆ-0.30% / -0.26%๏ผ‰๏ผŒๅพท่ฏญๅ›ฐๆƒ‘ๅบฆ็•ฅๆœ‰ไธŠๅ‡๏ผˆ8ๅ— +1.02%๏ผŒ40ๅ— +1.19%๏ผ‰ใ€‚่ฏญๆ–™ๅ“ˆๅธŒ่ง results.jsonใ€‚ๆจกๅž‹ๅค„ไบŽๅฎž้ชŒ็Šถๆ€๏ผŒไธๆไพ›้€š็”จ่ดจ้‡ไฟ่ฏใ€‚

ๅฎŒๆ•ดๆฅๆบใ€ๆ–‡ไปถๅˆ—่กจใ€ๆต‹่ฏ•ๆกไปถๅ’Œ้™ๅˆถ่งไธŠๆ–น่‹ฑๆ–‡่ฏดๆ˜Žใ€‚ๆจกๅž‹ๆƒ้‡ๅ—้™„ๅธฆ็š„ Qwen ่ฎธๅฏ็บฆๆŸ๏ผ›่ฟ่กŒๆ—ถ่ฝฏไปถ่ฎธๅฏไธ่ƒฝๆ›ฟไปฃๆจกๅž‹่ฎธๅฏใ€‚่ฟ™ๆ˜ฏ็‹ฌ็ซ‹็คพๅŒบ็‰ˆๆœฌ๏ผŒไธๆ˜ฏ Qwen ๆˆ– ISTA-DASLab ๅฎ˜ๆ–นๅ‘ๅธƒใ€‚

Downloads last month
516
GGUF
Model size
177B params
Architecture
qwen4exp
Hardware compatibility
Log In to add your hardware

2-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF

Quantized
(647)
this model

Collection including webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-Q2_0-512E-CP2-GGUF