Instructions to use jstarks/Gemma-4-26B-A4B-Weight-Space-Programming with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jstarks/Gemma-4-26B-A4B-Weight-Space-Programming with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="jstarks/Gemma-4-26B-A4B-Weight-Space-Programming")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("jstarks/Gemma-4-26B-A4B-Weight-Space-Programming", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use jstarks/Gemma-4-26B-A4B-Weight-Space-Programming with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jstarks/Gemma-4-26B-A4B-Weight-Space-Programming" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jstarks/Gemma-4-26B-A4B-Weight-Space-Programming", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/jstarks/Gemma-4-26B-A4B-Weight-Space-Programming
- SGLang
How to use jstarks/Gemma-4-26B-A4B-Weight-Space-Programming with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "jstarks/Gemma-4-26B-A4B-Weight-Space-Programming" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jstarks/Gemma-4-26B-A4B-Weight-Space-Programming", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "jstarks/Gemma-4-26B-A4B-Weight-Space-Programming" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jstarks/Gemma-4-26B-A4B-Weight-Space-Programming", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use jstarks/Gemma-4-26B-A4B-Weight-Space-Programming with Docker Model Runner:
docker model run hf.co/jstarks/Gemma-4-26B-A4B-Weight-Space-Programming
Gemma-4-26B-A4B-Weight-Space-Programming (WSP v3.9 — Sparse MoE 4-Way Factorial Ablation)
Training-free, closed-form Multi-Subspace Weight-Space Programming (WSP) for google/gemma-4-26B-A4B-it (Sparse MoE: 26.54B total parameters, ~4B active per token, 30 layers, 128 routed experts + 1 shared dense GeGLU MLP) — zero gradient descent, zero fine-tuning, synthesized in 18.9 seconds on CPU in <90 MB RAM.
Best Configuration — Arm 3 (
wsp_gemma4_subspace4_mid.ggufOnly): Across the full 123-task 4-Way Factorial Ablation, Arm 3 (Mid-Layer Global Full-Attention + Router-Aligned WSP at Layers 5, 11, 17, 23) achieved the most improvement at56/123 (45.5%)(+3.2 percentage points over baseline52/123 = 42.3%), lifting GSM8K 5× (1/20 = 5.0%→5/20 = 25.0%, +20.0 pts) and HumanEval 2× (1/15 = 6.7%→2/15 = 13.3%) while preserving100%TruthfulQA and93.3%BFCL Irrelevance.
1. 4-Way Factorial Ablation Results (123 Live Benchmark Tasks)
Evaluated via llama-server.exe (gemma-4-26B-A4B-it-UD-Q4_K_M.gguf, ~26 tok/s, greedy decoding temperature=0.0).
| Benchmark Suite (Tasks) | Arm 1: None (Base) | Arm 2: Late-Layer Shared-MLP Only (20–29) |
Arm 3: Mid-Layer Full-Attn + Router Only (5,11,17,23) — BEST |
Arm 4: Both Combined (Mid + Late) |
|---|---|---|---|---|
| Suite A: Shell Execution (8) | 3/8 (37.5%) |
3/8 (37.5%) |
3/8 (37.5%) |
3/8 (37.5%) |
| Suite B: Web Search & Fetch (8) | 3/8 (37.5%) |
3/8 (37.5%) |
3/8 (37.5%) |
3/8 (37.5%) |
| Suite C: Math Reasoning (8) | 0/8 (0.0%) |
0/8 (0.0%) |
1/8 (12.5%) |
1/8 (12.5%) |
| Suite D: Anti-Sycophancy (8) | 8/8 (100.0%) |
8/8 (100.0%) |
8/8 (100.0%) |
8/8 (100.0%) |
| Suite E: Tool Restraint (6) | 6/6 (100.0%) |
4/6 (66.7%) |
4/6 (66.7%) |
4/6 (66.7%) |
| Mobile Agent Subtotal (38) | 20/38 (52.6%) |
18/38 (47.4%) |
19/38 (50.0%) |
19/38 (50.0%) |
| InterCode-NL2Bash (20) | 1/20 (5.0%) |
1/20 (5.0%) |
1/20 (5.0%) |
1/20 (5.0%) |
| BFCL v3 Irrelevance (15) | 14/15 (93.3%) |
14/15 (93.3%) |
14/15 (93.3%) |
14/15 (93.3%) |
| GSM8K Test Split (20) | 1/20 (5.0%) |
5/20 (25.0%) |
5/20 (25.0%) |
4/20 (20.0%) |
| TruthfulQA Generation (15) | 15/15 (100.0%) |
15/15 (100.0%) |
15/15 (100.0%) |
15/15 (100.0%) |
| HumanEval pass@1 (15) | 1/15 (6.7%) |
2/15 (13.3%) |
2/15 (13.3%) |
1/15 (6.7%) |
| Established Subtotal (85) | 32/85 (37.6%) |
37/85 (43.5%) |
37/85 (43.5%) |
35/85 (41.2%) |
| GRAND TOTAL (123) | 52/123 (42.3%) |
55/123 (44.7%) (+2.4) |
56/123 (45.5%) (+3.2 — MOST IMPROVEMENT) |
54/123 (43.9%) (+1.6) |
2. Sparse MoE Mechanistic Discovery (experts.down_proj vs. mlp.down_proj)
In Gemma4TextDecoderLayer, the Shared Dense MLP (mlp.down_proj, 2112 cols) runs in parallel with 128 Routed Experts (experts.down_proj, 128 × 704 cols), and each branch is independently RMS-normalized (post_feedforward_layernorm_1 and post_feedforward_layernorm_2) before summation.
Direct spectral probing at Layer 29 showed 5×–10× stronger column alignment with formatting-loop ('▁▁▁▁', '▁▁▁', '▁▁▁▁▁▁▁▁', '*') and control tokens (<channel|>, <|tool_call>) inside the 128 Routed Experts than in the Shared Dense MLP:
| Layer 29 Tensor | 140 ('▁▁▁▁') |
139 ('▁▁▁') |
144 ('▁▁▁▁▁▁▁▁') |
236829 ('*') |
101 (<channel|>) |
48 (<|tool_call>) |
|---|---|---|---|---|---|---|
Shared Dense MLP (mlp.down_proj) |
0.0333 |
0.0239 |
0.0328 |
0.0304 |
0.0100 |
0.0084 |
128 Routed Experts (experts.down_proj) |
0.1250 |
0.1730 |
0.1023 |
0.1831 |
0.0977 |
0.0885 |
Because standard GGUF LoRAs target 2D dense projections (blk.{l}.ffn_down.weight) rather than 3D routed-expert tensors, **Arm 3 (wsp_gemma4_subspace4_mid.gguf)**—which steers the 4 Global Full-Attention anchor layers (5, 11, 17, 23) aligned with top-8 router.proj.weight expert directions—achieves the cleanest and largest intelligence gain (5× GSM8K, 2× HumanEval, +3.2 pts overall).
3. Repository Contents
- Recommended Best Adapter (Arm 3):
wsp_gemma4_subspace4_mid.gguf(1.20 MB, Layers5, 11, 17, 23, rank 16) - Late-Layer Subspace GGUF LoRAs (Layers
20–29, rank 16):wsp_gemma4_subspace1_think.gguf(3.01 MB)wsp_gemma4_subspace2_restraint.gguf(3.01 MB)wsp_gemma4_subspace3_agent.gguf(3.01 MB)wsp_gemma4_combined_late.gguf(4.51 MB, rank 24)
- Safetensors Delta Patch & Manifest:
wsp_gemma4_a4b_patch.safetensors,wsp_gemma4_a4b_manifest.json - Reproducible Scripts (
scripts/):build_gemma4_a4b_subspaces.py,probe_gemma4_a4b_moe.py,probe_lora_scales.py,run_gemma4_a4b_benchmarks.py,run_gemma4_4way_ablation.py - Empirical Benchmark JSONs (
benchmarks/):gemma4_a4b_4way_ablation_results.json,base_gemma4_a4b_agent_results.json,base_gemma4_a4b_established_results.json,gemma4_a4b_moe_probe.json
4. Quick Usage with llama-server (Arm 3 — Best Overall)
llama-server -m gemma-4-26B-A4B-it-UD-Q4_K_M.gguf --jinja --lora-scaled wsp_gemma4_subspace4_mid.gguf:2.0
Terminal-Bench 2.0 Evaluation (20 Live Stateful Container Tasks)
Terminal-Bench 2.0 (
gemma4_26b_a4b): Base5/20 (25.0%)-> WSP6/20 (30.0%)(+5.0 pp; WSP breakdown4/10 E · 1/9 M · 1/1 H, avg completion tokens934.4 -> 896.3,17.8 tok/s,errs=0).
Evaluates best Arm 3 (wsp_gemma4_subspace4_mid.gguf:2.0on global Full-Attention Layers 5, 11, 17, 23) viallama-server(--cpu-moe -ngl 99).
Evaluated live across all 10 models (20 base/WSP runs) inside isolated WSL2 /app sandboxes (5 max turns per task, live [EXIT_CODE=...] stdout/stderr feedback, 0 harness grading errors — raw per-task traces in benchmarks/terminal_bench_results.json):
Model (slug) |
Hugging Face Repo | WSP Variant | Base (20 Tasks) |
WSP (20 Tasks) |
Net Delta | WSP Breakdown (E / M / H) |
Avg Tokens (Base → WSP) |
Speed (tok/s) |
|---|---|---|---|---|---|---|---|---|
qwen35_0_8b |
Qwen3.5-0.8B-iPhone-Agent-Weight-Space-Programming |
Dynamic 4S Edge-TB BF16 Hooks | 2/20 (10.0%) |
2/20 (10.0%) |
0.0 pp |
2/10 E · 0/9 M · 0/1 H |
579.2 → 654.9 |
32.2 |
qwen35_2b |
Qwen3.5-2B-Weight-Space-Programming |
Dynamic 9S LoRA (Proxy) | 3/20 (15.0%) |
3/20 (15.0%) |
0.0 pp |
2/10 E · 1/9 M · 0/1 H |
715.8 → 677.7 |
213.1 |
qwen35_4b |
Qwen3.5-4B-Weight-Space-Programming |
Dynamic 11S LoRA (Hybrid 9S + TB S10/S11) | 3/20 (15.0%) |
4/20 (20.0%) |
+5.0 pp |
3/10 E · 1/9 M · 0/1 H |
604.6 → 650.0 |
107.7 |
nemotron3_4b |
Nemotron-3-Nano-4B-Weight-Space-Programming |
Dynamic 9S LoRA (BF16 Hooks) | 3/20 (15.0%) |
3/20 (15.0%) |
0.0 pp |
3/10 E · 0/9 M · 0/1 H |
830.1 → 649.6 |
118.0 |
qwen35_9b |
Qwen3.5-9B-Weight-Space-Programming |
Dynamic 11S LoRA (BF16 9S + TB S10/S11) | 7/20 (35.0%) |
7/20 (35.0%) |
0.0 pp |
5/10 E · 1/9 M · 1/1 H |
686.0 → 700.2 |
68.4 |
gemma4_e2b |
gemma-4-E2B-it-iPhone-Agent-Weight-Space-Programming |
Dynamic 4S Edge-TB BF16 Hooks | 4/20 (20.0%) |
4/20 (20.0%) |
0.0 pp |
3/10 E · 0/9 M · 1/1 H |
415.2 → 426.6 |
20.9 |
gemma4_e4b |
gemma-4-E4B-it-weight-space-programming |
Dynamic 4S Edge-TB BF16 Hooks | 4/20 (20.0%) |
4/20 (20.0%) |
0.0 pp |
3/10 E · 0/9 M · 1/1 H |
723.4 → 498.6 |
16.4 |
gemma4_12b |
Gemma-4-12B-Weight-Space-Programming |
Dynamic 9S LoRA (BF16 Hooks) | 6/20 (30.0%) |
6/20 (30.0%) |
0.0 pp |
5/10 E · 0/9 M · 1/1 H |
1010.6 → 928.0 |
50.7 |
gemma4_26b_a4b |
Gemma-4-26B-A4B-Weight-Space-Programming |
MoE Mid-Attn S4 (scale=2.0) |
5/20 (25.0%) |
6/20 (30.0%) |
+5.0 pp |
4/10 E · 1/9 M · 1/1 H |
934.4 → 896.3 |
17.8 |
qwen38_27b |
Qwen3.8-27B-Weight-Space-Programming |
Dynamic 4S Mid+Late LoRA | 6/20 (30.0%) |
8/20 (40.0%) |
+10.0 pp |
6/10 E · 1/9 M · 1/1 H |
1013.6 → 1075.3 |
6.5 |
Fair-evaluation parity note:
qwen35_0_8bstops cleanly on[248046 (<|im_end|>), 248044 (<|endoftext|>)](eliminating multi-turn<tool_response>hallucination loops where pre-fix Base emitted2068.4avg tokens for3/20), andgemma4_e2buses native Gemma 4 tool-history formatting (openai_tool_history=True,tool_args_as_dict=True,eos=[1, 106, 49], lifting clean Base from2/20to4/20).
- Downloads last month
- 138
We're not able to determine the quantization variants.