--- license: apache-2.0 base_model: nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 tags: - weight-space-programming - mechanistic-interpretability - gguf - lora - safetensors - dynamic-lora - llama-cpp - tool-calling - agent - gsm8k - humaneval - bfcl - nl2bash - terminal-bench - nemotron - nemotron-h - mamba2 - hybrid-ssm pipeline_tag: text-generation library_name: transformers --- # Nemotron-3-Nano-4B-Weight-Space-Programming (WSP v3.0 — 9-Subspace Dynamic LoRA Architecture) **Training-free, closed-form 9-Subspace Weight-Space Programming (WSP v3.0) for [`nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16) (`42` layers, `hidden_size=3136`, `intermediate_size=12544`, `relu2` MLP, `w` RMSNorm, `synthesis_mode=forward_hooks_bf16`), evaluated end-to-end across `100` unified benchmark tasks (`30` Disjoint Holdout across `S1..S9` + `70` Established Public Benchmarks) and `20` live stateful Terminal-Bench 2.0 container tasks in WSL2.** > [!IMPORTANT] > **Uniform 100-Task + Terminal-Bench 2.0 Result (`nemotron3_4b`):** > - **Unified 100-Task Suite:** Base `81/100 (81.0%)` -> **WSP v3.0 `81/100 (81.0%)`** (**`+0.0 pp`**) > - **Holdout (`S1..S9`, `30` tasks):** `26/30 -> 28/30` > - **Established Public (`70` tasks: `20` GSM8K, `15` HumanEval, `15` BFCL Irrelevance, `20` NL2Bash):** `55/70 -> 53/70` > - **Terminal-Bench 2.0 (`20` live WSL2 bash tasks):** Base `3/20 (15.0%)` -> **WSP `3/20 (15.0%)`** (**`0.0 pp`**; `118.0 tok/s`) > - **Speed & VRAM (`-c 16384 -ctk q8_0 -ctv q8_0`):** `131.1 tok/s`, load delta `2999 MiB`, peak delta **`3133 MiB`** > - **Cross-Hardware Fit:** RTX 2060 (`6 GB`): **Yes (~3.0 GiB spare @ 16k)** | RTX 4070 Ti (`12 GB`): **Yes (fastest 4B @ 16k)** --- ## 1. Per-Suite 100-Task Breakdown (`Base` -> `WSP v3.0`) | Suite | Tasks | Base Pass | **WSP v3.0 Pass** | Avg Completion Tokens (`Base → WSP`) | |---|---:|:---:|:---:|:---:| | `S1_THINK` (Holdout S1: Think Closure) | `3` | `3/3 (100.0%)` | **`3/3 (100.0%)`** | `223.3 → 144.3` | | `S2_RESTRAINT` (Holdout S2: Tool Restraint & Anti-Sycophancy) | `4` | `3/4 (75.0%)` | **`4/4 (100.0%)`** | `261.8 → 249.2` | | `S3_ROUTER` (Holdout S3: Multi-Tool Router) | `5` | `5/5 (100.0%)` | **`5/5 (100.0%)`** | `79.8 → 95.0` | | `S4_CODE` (Holdout S4: Mid-Band Code Synthesis) | `2` | `2/2 (100.0%)` | **`2/2 (100.0%)`** | `690.5 → 577.0` | | `S5_TURN2_JSON` (Holdout S5: Turn-2 Structured JSON) | `3` | `3/3 (100.0%)` | **`3/3 (100.0%)`** | `191.3 → 224.0` | | `S6_HANDOFF` (Holdout S6: Escalation Handoff Gate) | `4` | `4/4 (100.0%)` | **`4/4 (100.0%)`** | `117.5 → 137.2` | | `S7_INJ_GUARD` (Holdout S7: Indirect Injection Guard) | `3` | `2/3 (66.7%)` | **`2/3 (66.7%)`** | `184.3 → 178.3` | | `S8_GROUNDING` (Holdout S8: Grounded Abstention) | `3` | `2/3 (66.7%)` | **`3/3 (100.0%)`** | `143.0 → 232.7` | | `S9_ERR_RECOV` (Holdout S9: Error Self-Recovery) | `3` | `2/3 (66.7%)` | **`2/3 (66.7%)`** | `76.7 → 89.0` | | `EST_GSM8K` (Established: OpenAI GSM8K (20)) | `20` | `19/20 (95.0%)` | **`19/20 (95.0%)`** | `215.7 → 227.9` | | `EST_HUMANEVAL` (Established: HumanEval pass@1 (15)) | `15` | `13/15 (86.7%)` | **`14/15 (93.3%)`** | `379.1 → 317.8` | | `EST_BFCL_IRR` (Established: BFCL v3 Irrelevance (15)) | `15` | `10/15 (66.7%)` | **`10/15 (66.7%)`** | `339.1 → 339.1` | | `EST_NL2BASH` (Established: InterCode-NL2Bash (20)) | `20` | `13/20 (65.0%)` | **`10/20 (50.0%)`** | `302.6 → 308.6` | --- ## 2. All 5 Models Evaluated Uniformly (`100` Tasks Each) Every candidate in the `wsp-5model-pipeline` study was evaluated under identical decoding (`temperature=0.0, seed=42`, `-c 16384 -ctk q8_0 -ctv q8_0`) across all **30 disjoint `S1..S9` holdout tasks** and all **70 cached established tasks**: | Model (`slug`) | Hugging Face Repo | GGUF (`Q4_K_M`) | Base (`100` Tasks) | **WSP v3.0 (`100` Tasks)** | Net Delta | Holdout (`/30`) | Est. (`/70`) | Speed (`tok/s`) | Peak VRAM Delta (`16k`) | RTX 2060 (`6 GB`) | RTX 4070 Ti (`12 GB`) | |---|---|---|---|---|---|---|---|---|---|---|---| | **`qwen35_4b`** | [`Qwen3.5-4B-Weight-Space-Programming`](https://huggingface.co/jstarks/Qwen3.5-4B-Weight-Space-Programming) | `2.553 GiB` | `72/100 (72.0%)` | **`89/100 (89.0%)`** | **`+17.0 pp`** | `24 → 29/30` | `48 → 60/70` | `117.0 tok/s` | `3370 MiB` | Yes (~2.7 GiB spare @ 16k) | Yes (~8.0 GiB spare @ 16k) | | **`nemotron3_4b`** | [`Nemotron-3-Nano-4B-Weight-Space-Programming`](https://huggingface.co/jstarks/Nemotron-3-Nano-4B-Weight-Space-Programming) | `2.642 GiB` | `81/100 (81.0%)` | **`81/100 (81.0%)`** | **`+0.0 pp`** | `26 → 28/30` | `55 → 53/70` | `131.1 tok/s` | `3133 MiB` | Yes (~3.0 GiB spare @ 16k) | Yes (fastest 4B @ 16k) | | **`qwen35_9b`** | [`Qwen3.5-9B-Weight-Space-Programming`](https://huggingface.co/jstarks/Qwen3.5-9B-Weight-Space-Programming) | `5.290 GiB` | `84/100 (84.0%)` | **`88/100 (88.0%)`** | **`+4.0 pp`** | `26 → 28/30` | `58 → 60/70` | `72.5 tok/s` | `5650 MiB` | Tight (~5.3 GiB @ 4k before desktop) | Yes (16k solo or 8k cascade Tier-2) | | **`qwen35_2b`** | [`Qwen3.5-2B-Weight-Space-Programming`](https://huggingface.co/jstarks/Qwen3.5-2B-Weight-Space-Programming) | `1.193 GiB` | `57/100 (57.0%)` | **`55/100 (55.0%)`** | **`-2.0 pp`** | `23 → 23/30` | `34 → 32/70` | `226.5 tok/s` | `1979 MiB` | Yes (~4.4 GiB spare @ 16k) | Yes (ultra-light leaf / cascade Tier-1) | | **`gemma4_12b`** | [`Gemma-4-12B-Weight-Space-Programming`](https://huggingface.co/jstarks/Gemma-4-12B-Weight-Space-Programming) | `6.633 GiB` | `50/100 (50.0%)` | **`52/100 (52.0%)`** | **`+2.0 pp`** | `23 → 24/30` | `27 → 28/70` | `51.3 tok/s` | `7664 MiB` | No (> 6144 MiB VRAM) | Yes (16k solo; ~3.8 GiB free) | --- ## 3. Terminal-Bench 2.0 Cross-Model Evaluation (All 10 Models, 20 Stateful Tasks) Evaluated live inside clean per-task `/app` sandboxes in WSL2 (`5` max turns, `[EXIT_CODE=...]` feedback, zero grading errors across all `20` runs — full per-task traces in `benchmarks/terminal_bench_results.json`): | Model (`slug`) | Hugging Face Repo | WSP Variant | Base (`20` Tasks) | **WSP (`20` Tasks)** | Net Delta | WSP Breakdown (`E / M / H`) | Avg Tokens (`Base → WSP`) | Speed (`tok/s`) | |---|---|---|---|---|---|---|---|---| | **`qwen35_0_8b`** | [`Qwen3.5-0.8B-iPhone-Agent-Weight-Space-Programming`](https://huggingface.co/jstarks/Qwen3.5-0.8B-iPhone-Agent-Weight-Space-Programming) | Dynamic 4S Edge-TB BF16 Hooks | `2/20 (10.0%)` | **`2/20 (10.0%)`** | **`0.0 pp`** | `2/10 E · 0/9 M · 0/1 H` | `579.2 → 654.9` | `32.2` | | **`qwen35_2b`** | [`Qwen3.5-2B-Weight-Space-Programming`](https://huggingface.co/jstarks/Qwen3.5-2B-Weight-Space-Programming) | Dynamic 9S LoRA (Proxy) | `3/20 (15.0%)` | **`3/20 (15.0%)`** | **`0.0 pp`** | `2/10 E · 1/9 M · 0/1 H` | `715.8 → 677.7` | `213.1` | | **`qwen35_4b`** | [`Qwen3.5-4B-Weight-Space-Programming`](https://huggingface.co/jstarks/Qwen3.5-4B-Weight-Space-Programming) | Dynamic 11S LoRA (Hybrid 9S + TB S10/S11) | `3/20 (15.0%)` | **`4/20 (20.0%)`** | **`+5.0 pp`** | `3/10 E · 1/9 M · 0/1 H` | `604.6 → 650.0` | `107.7` | | **`nemotron3_4b`** | [`Nemotron-3-Nano-4B-Weight-Space-Programming`](https://huggingface.co/jstarks/Nemotron-3-Nano-4B-Weight-Space-Programming) | Dynamic 9S LoRA (BF16 Hooks) | `3/20 (15.0%)` | **`3/20 (15.0%)`** | **`0.0 pp`** | `3/10 E · 0/9 M · 0/1 H` | `830.1 → 649.6` | `118.0` | | **`qwen35_9b`** | [`Qwen3.5-9B-Weight-Space-Programming`](https://huggingface.co/jstarks/Qwen3.5-9B-Weight-Space-Programming) | Dynamic 11S LoRA (BF16 9S + TB S10/S11) | `7/20 (35.0%)` | **`7/20 (35.0%)`** | **`0.0 pp`** | `5/10 E · 1/9 M · 1/1 H` | `686.0 → 700.2` | `68.4` | | **`gemma4_e2b`** | [`gemma-4-E2B-it-iPhone-Agent-Weight-Space-Programming`](https://huggingface.co/jstarks/gemma-4-E2B-it-iPhone-Agent-Weight-Space-Programming) | Dynamic 4S Edge-TB BF16 Hooks | `4/20 (20.0%)` | **`4/20 (20.0%)`** | **`0.0 pp`** | `3/10 E · 0/9 M · 1/1 H` | `415.2 → 426.6` | `20.9` | | **`gemma4_e4b`** | [`gemma-4-E4B-it-weight-space-programming`](https://huggingface.co/jstarks/gemma-4-E4B-it-weight-space-programming) | Dynamic 4S Edge-TB BF16 Hooks | `4/20 (20.0%)` | **`4/20 (20.0%)`** | **`0.0 pp`** | `3/10 E · 0/9 M · 1/1 H` | `723.4 → 498.6` | `16.4` | | **`gemma4_12b`** | [`Gemma-4-12B-Weight-Space-Programming`](https://huggingface.co/jstarks/Gemma-4-12B-Weight-Space-Programming) | Dynamic 9S LoRA (BF16 Hooks) | `6/20 (30.0%)` | **`6/20 (30.0%)`** | **`0.0 pp`** | `5/10 E · 0/9 M · 1/1 H` | `1010.6 → 928.0` | `50.7` | | **`gemma4_26b_a4b`** | [`Gemma-4-26B-A4B-Weight-Space-Programming`](https://huggingface.co/jstarks/Gemma-4-26B-A4B-Weight-Space-Programming) | MoE Mid-Attn S4 (`scale=2.0`) | `5/20 (25.0%)` | **`6/20 (30.0%)`** | **`+5.0 pp`** | `4/10 E · 1/9 M · 1/1 H` | `934.4 → 896.3` | `17.8` | | **`qwen38_27b`** | [`Qwen3.8-27B-Weight-Space-Programming`](https://huggingface.co/jstarks/Qwen3.8-27B-Weight-Space-Programming) | Dynamic 4S Mid+Late LoRA | `6/20 (30.0%)` | **`8/20 (40.0%)`** | **`+10.0 pp`** | `6/10 E · 1/9 M · 1/1 H` | `1013.6 → 1075.3` | `6.5` | > *Fair-evaluation parity note:* `qwen35_0_8b` stops cleanly on `[248046 (<|im_end|>), 248044 (<|endoftext|>)]` (eliminating multi-turn `` hallucination loops where pre-fix Base emitted `2068.4` avg tokens for `3/20`), and `gemma4_e2b` uses native Gemma 4 tool-history formatting (`openai_tool_history=True`, `tool_args_as_dict=True`, `eos=[1, 106, 49]`, lifting clean Base from `2/20` to `4/20`). --- ## 4. Synthesized Rank-16 `f32` Subspaces (`S1..S9`) & Safetensors Patches - **GGUF LoRA Adapters (`llama-server`):** `adapters/wsp_s1_think.gguf .. wsp_s9_error_recovery.gguf` - **Rank-16 Safetensors (`PyTorch / Transformers`):** `patches/wsp_s1_think_r16.safetensors .. wsp_s9_error_recovery_r16.safetensors` - **Combined Unit-Sum Dense BF16 Delta Patch:** `patches/wsp_nemotron3_4b_patch.safetensors` (`675.28 MiB`) | Slot | GGUF LoRA (`llama.cpp`) | Safetensors (`PyTorch`) | Target Layers | `\|\|\Delta W\|\|_F` | Cross Leak | Syntax Leak | Role | |:---:|---|---|---|---:|---:|---:|---| | `id=0` | `adapters/wsp_s1_think.gguf` | `patches/wsp_s1_think_r16.safetensors` (`4.8 MiB`) | `[29, 33, 37, 39, 41]` | `11.2290` | `0.145%` | `0.024%` | Fast `` / `` closure & reasoning compression | | `id=1` | `adapters/wsp_s2_restraint.gguf` | `patches/wsp_s2_restraint_r16.safetensors` (`4.8 MiB`) | `[29, 33, 37, 39, 41]` | `14.4057` | `0.181%` | `0.023%` | Tool restraint & anti-sycophancy pushback | | `id=2` | `adapters/wsp_s3_router.gguf` | `patches/wsp_s3_router_r16.safetensors` (`4.8 MiB`) | `[29, 33, 37, 39, 41]` | `8.1335` | `0.132%` | `0.040%` | Multi-tool boundary router (`exec`, `web_search`, `read_file`, `write_file`) | | `id=3` | `adapters/wsp_s4_mid_code.gguf` | `patches/wsp_s4_mid_code_r16.safetensors` (`3.8 MiB`) | `[13, 18, 22, 25]` | `1.7745` | `0.136%` | `0.177%` | Mid-network algorithmic code synthesis | | `id=4` | `adapters/wsp_s5_json_schema.gguf` | `patches/wsp_s5_json_schema_r16.safetensors` (`4.8 MiB`) | `[29, 33, 37, 39, 41]` | `9.1762` | `0.076%` | `0.036%` | Turn-2 structured JSON synthesis gate with tools attached | | `id=5` | `adapters/wsp_s6_handoff.gguf` | `patches/wsp_s6_handoff_r16.safetensors` (`4.8 MiB`) | `[29, 33, 37, 39, 41]` | `14.9598` | `0.171%` | `0.026%` | Bounded subtask vs. unrecoverable `escalate_handoff` gate | | `id=6` | `adapters/wsp_s7_injection_guard.gguf` | `patches/wsp_s7_injection_guard_r16.safetensors` (`4.8 MiB`) | `[29, 33, 37, 39, 41]` | `11.4327` | `0.155%` | `0.031%` | Indirect prompt-injection defense inside tool responses | | `id=7` | `adapters/wsp_s8_grounding.gguf` | `patches/wsp_s8_grounding_r16.safetensors` (`4.8 MiB`) | `[29, 33, 37, 39, 41]` | `10.2065` | `0.103%` | `0.013%` | Grounded context fidelity & missing-field abstention | | `id=8` | `adapters/wsp_s9_error_recovery.gguf` | `patches/wsp_s9_error_recovery_r16.safetensors` (`4.8 MiB`) | `[29, 33, 37, 39, 41]` | `11.6176` | `0.117%` | `0.016%` | Recoverable local tool-error self-correction | --- ## 5. Quickstart on Any Machine (`run_wsp.py` / OpenAI Proxy / Terminal Chat) ### Option A: One-Command Auto-Download + Dynamic 9-Subspace Proxy Works on any Windows, Linux, or macOS computer with Python + [llama.cpp (`llama-server`)](https://github.com/ggml-org/llama.cpp): ```bash pip install huggingface_hub hf download jstarks/Nemotron-3-Nano-4B-Weight-Space-Programming run_wsp.py --local-dir ./nemotron3_4b-wsp python ./nemotron3_4b-wsp/run_wsp.py --chat ``` Omit `--chat` to expose an OpenAI-compatible dynamic routing proxy on `http://127.0.0.1:8090/v1` (with `/health` and `/v1/chat/completions`), or pass `--download-only` to download only the GGUF + 9 GGUF LoRAs + metadata (`--include-bf16` also fetches safetensors patches / HF files). ### Option B: Manual `llama-server` + `scripts/dynamic_router_9s.py` ```bash llama-server -m NVIDIA-Nemotron3-Nano-4B-Q4_K_M.gguf --host 127.0.0.1 --port 8089 \ -ngl 99 -np 1 -c 16384 -ctk q8_0 -ctv q8_0 -b 512 -ub 256 \ --jinja --reasoning-format deepseek --lora-init-without-apply --lora adapters/wsp_s1_think.gguf --lora adapters/wsp_s2_restraint.gguf --lora adapters/wsp_s3_router.gguf --lora adapters/wsp_s4_mid_code.gguf --lora adapters/wsp_s5_json_schema.gguf --lora adapters/wsp_s6_handoff.gguf --lora adapters/wsp_s7_injection_guard.gguf --lora adapters/wsp_s8_grounding.gguf --lora adapters/wsp_s9_error_recovery.gguf ``` ### Option C: PyTorch / Hugging Face In-Memory Patching (`scripts/apply_patch_9s.py`) ```python from transformers import AutoModelForCausalLM from scripts.apply_patch_9s import REGIME_PRESETS, apply_subspace_scales_to_hf_model model = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16", torch_dtype="bfloat16") apply_subspace_scales_to_hf_model(model, "patches", REGIME_PRESETS["general_fast_think"]) ```