Instructions to use jstarks/gemma-4-E4B-it-weight-space-programming with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jstarks/gemma-4-E4B-it-weight-space-programming with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="jstarks/gemma-4-E4B-it-weight-space-programming") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("jstarks/gemma-4-E4B-it-weight-space-programming") model = AutoModelForMultimodalLM.from_pretrained("jstarks/gemma-4-E4B-it-weight-space-programming", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use jstarks/gemma-4-E4B-it-weight-space-programming with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jstarks/gemma-4-E4B-it-weight-space-programming" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jstarks/gemma-4-E4B-it-weight-space-programming", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/jstarks/gemma-4-E4B-it-weight-space-programming
- SGLang
How to use jstarks/gemma-4-E4B-it-weight-space-programming with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "jstarks/gemma-4-E4B-it-weight-space-programming" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jstarks/gemma-4-E4B-it-weight-space-programming", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "jstarks/gemma-4-E4B-it-weight-space-programming" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jstarks/gemma-4-E4B-it-weight-space-programming", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use jstarks/gemma-4-E4B-it-weight-space-programming with Docker Model Runner:
docker model run hf.co/jstarks/gemma-4-E4B-it-weight-space-programming
Use Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "jstarks/gemma-4-E4B-it-weight-space-programming" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "jstarks/gemma-4-E4B-it-weight-space-programming",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Describe this image in one sentence."
},
{
"type": "image_url",
"image_url": {
"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
}
}
]
}
]
}'
Gemma-4-E4B-it — Weight-Space Programming Patch (0.002% Weights Edited, Zero Training)
This repository provides a direct weight-space edit for Google's gemma-4-E4B-it (8B total parameters / 4B active text parameters), inspired by Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights (Takehi et al., MIT CSAIL).
Without a single step of gradient descent, backpropagation, or fine-tuning, modifying 64 MLP down_proj columns across Layers 38–41 (163,840 parameters = 0.00205% of the model) resolves two critical pathologies in Gemma 4 E4B's <|think|> mode:
- Thinking + Tool-Calling Paralysis (
0.0%→75.0%): In baselinegemma-4-E4B-it, enabling<|think|>(enable_thinking=True) causes the model to either get trapped in 256+ token verbose markdown outlines (1. **Analyze the Request:** ...) or hallucinate temporal refusals ("Since 2026 is in the future...") instead of invoking declared tools likeweb_search. - Multi-Step Math Token Runaway (
33.3%→66.7%): Suppresses the meta-commentary outline loop ("Here's a thinking process to solve this problem: 1. **Identify...") so the model jumps directly into algebraic derivation ("Let N be...").
Before vs. After Benchmark Results
Full per-prompt transcripts, parsed tool calls, latencies, and column indices are included in benchmark_before_after.json.
| Benchmark Suite | Metric | Baseline (Before) | Weight-Space Patched (After) | Delta (Δ) |
|---|---|---|---|---|
| Model Modification | Parameters Edited (bfloat16) |
0 (0.0%) |
163,840 (0.00205%) |
64 MLP cols (L38–L41) |
| Suite A: Thinking + Web Search (8 queries) | Valid Tool-Call Rate | 0.0% (0/8) |
75.0% (6/8) |
+75.0% |
| Suite A: Efficiency | Avg Tokens / Avg Latency (RTX 4070 Ti) | 243.8 tok / 15.46s |
129.0 tok / 7.85s |
-47.1% tokens / 1.97x faster |
| Suite B: Multi-Step Math (Thinking) (6 problems) | Solved Within 420-Token Budget | 33.3% (2/6) |
66.7% (4/6) |
+33.4% (2x) |
| Suite C: Direct Knowledge + Restraint (6 prompts) | Tool Restraint (No False Search) | 100.0% (6/6) |
100.0% (6/6) |
0.0% degradation |
| Suite D: Live DuckDuckGo Agent Loop (3 E2E runs) | End-to-End Grounded Synthesis | 0.0% (0/3) |
66.7% (2/3) |
+66.7% |
Quickstart: Two Ways to Use
Option A: Load the Full Standalone Model Directly (transformers)
This repository contains the complete 16 GB bfloat16 model.safetensors, config.json, tokenizer.json, and chat_template.jinja, so you can load it directly via transformers or our 12 GB VRAM-optimized loader:
from transformers import AutoProcessor, AutoModelForMultimodalLM
import torch
model_id = "jstarks/gemma-4-E4B-it-weight-space-programming"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
model_id, device_map="auto", torch_dtype=torch.bfloat16
)
Or on a 12 GB GPU (like an RTX 4070 Ti) using the included model_loader.py (which keeps the 42 active layers on GPU in 9.29 GB VRAM and offloads the 5.64 GB static Per-Layer Embedding table to CPU):
from model_loader import load_gemma4_e4b_text
model, tokenizer = load_gemma4_e4b_text("jstarks/gemma-4-E4B-it-weight-space-programming")
Option B: Apply the 641 KB Instant Patch to an Existing Checkpoint in <1 Second
If you already have google/gemma-4-E4B-it or unsloth/gemma-4-E4B-it downloaded locally, you can apply gemma4_e4b_weight_space_patch.safetensors (641 KB) in-memory without downloading 16 GB:
from apply_patch import apply_gemma4_weight_space_patch
from model_loader import load_gemma4_e4b_text
# 1. Load your existing Gemma-4-E4B-it checkpoint
model, tokenizer = load_gemma4_e4b_text("unsloth/gemma-4-E4B-it")
# 2. Apply the 64-column weight-space patch (163,840 parameters across Layers 38-41)
apply_gemma4_weight_space_patch(model, "gemma4_e4b_weight_space_patch.safetensors")
Key Mechanistic Findings on Gemma 4 E4B
- The Layer 23 Boundary Hub (
post_ffn_norm = 26.16,layer_scalar = 0.0654):- In
gemma-4-E4B-it, Layers 0–23 have independent KV heads while Layers 24–41 (num_kv_shared_layers = 18) share KV states. At the boundary (Layer 23, a globalfull_attentionlayer),post_feedforward_layernormhas a mean weight of26.16(compared to0.5–2.7in other layers), but its residual output is immediately multiplied by a learned scalar bufferlayer_scalar = 0.0654.
- In
- RMSNorm-Preconditioned Surgery in Layers 38–41:
- Because each Gemma 4 decoder layer passes
mlp(x)throughpost_feedforward_layernormbefore adding to the residual stream, modifyingdown_projcolumns in Layers 38–41 requires preconditioning the target residual directiond_targetbymax(|w_post|, 0.25)^(-1)so high-norm outlier coordinates inw_postdo not distort the vocabulary projection.
- Because each Gemma 4 decoder layer passes
Terminal-Bench 2.0 Evaluation (20 Live Stateful Container Tasks)
Terminal-Bench 2.0 (
gemma4_e4b): Base4/20 (20.0%)-> WSP4/20 (20.0%)(0.0 pp; WSP breakdown3/10 E · 0/9 M · 1/1 H, avg completion tokens723.4 -> 498.6,16.4 tok/s,errs=0).
Evaluates the 4-subspace rank-16 Edge-TB BF16 adapters (adapters_tbench/edge_tbench_adapters.safetensorson Mid[11,17,23]+ Late[35,38,39,40,41],7.04 MB,edge_wsp_router.py) inside the 5-turn WSL2 Terminal-Bench loop (3/20 -> 4/20, unlockshard=1/1,-31.1%tokens vs Base723.4 -> 498.6).
Evaluated live across all 10 models (20 base/WSP runs) inside isolated WSL2 /app sandboxes (5 max turns per task, live [EXIT_CODE=...] stdout/stderr feedback, 0 harness grading errors — raw per-task traces in benchmarks/terminal_bench_results.json):
Model (slug) |
Hugging Face Repo | WSP Variant | Base (20 Tasks) |
WSP (20 Tasks) |
Net Delta | WSP Breakdown (E / M / H) |
Avg Tokens (Base → WSP) |
Speed (tok/s) |
|---|---|---|---|---|---|---|---|---|
qwen35_0_8b |
Qwen3.5-0.8B-iPhone-Agent-Weight-Space-Programming |
Dynamic 4S Edge-TB BF16 Hooks | 2/20 (10.0%) |
2/20 (10.0%) |
0.0 pp |
2/10 E · 0/9 M · 0/1 H |
579.2 → 654.9 |
32.2 |
qwen35_2b |
Qwen3.5-2B-Weight-Space-Programming |
Dynamic 9S LoRA (Proxy) | 3/20 (15.0%) |
3/20 (15.0%) |
0.0 pp |
2/10 E · 1/9 M · 0/1 H |
715.8 → 677.7 |
213.1 |
qwen35_4b |
Qwen3.5-4B-Weight-Space-Programming |
Dynamic 11S LoRA (Hybrid 9S + TB S10/S11) | 3/20 (15.0%) |
4/20 (20.0%) |
+5.0 pp |
3/10 E · 1/9 M · 0/1 H |
604.6 → 650.0 |
107.7 |
nemotron3_4b |
Nemotron-3-Nano-4B-Weight-Space-Programming |
Dynamic 9S LoRA (BF16 Hooks) | 3/20 (15.0%) |
3/20 (15.0%) |
0.0 pp |
3/10 E · 0/9 M · 0/1 H |
830.1 → 649.6 |
118.0 |
qwen35_9b |
Qwen3.5-9B-Weight-Space-Programming |
Dynamic 11S LoRA (BF16 9S + TB S10/S11) | 7/20 (35.0%) |
7/20 (35.0%) |
0.0 pp |
5/10 E · 1/9 M · 1/1 H |
686.0 → 700.2 |
68.4 |
gemma4_e2b |
gemma-4-E2B-it-iPhone-Agent-Weight-Space-Programming |
Dynamic 4S Edge-TB BF16 Hooks | 4/20 (20.0%) |
4/20 (20.0%) |
0.0 pp |
3/10 E · 0/9 M · 1/1 H |
415.2 → 426.6 |
20.9 |
gemma4_e4b |
gemma-4-E4B-it-weight-space-programming |
Dynamic 4S Edge-TB BF16 Hooks | 4/20 (20.0%) |
4/20 (20.0%) |
0.0 pp |
3/10 E · 0/9 M · 1/1 H |
723.4 → 498.6 |
16.4 |
gemma4_12b |
Gemma-4-12B-Weight-Space-Programming |
Dynamic 9S LoRA (BF16 Hooks) | 6/20 (30.0%) |
6/20 (30.0%) |
0.0 pp |
5/10 E · 0/9 M · 1/1 H |
1010.6 → 928.0 |
50.7 |
gemma4_26b_a4b |
Gemma-4-26B-A4B-Weight-Space-Programming |
MoE Mid-Attn S4 (scale=2.0) |
5/20 (25.0%) |
6/20 (30.0%) |
+5.0 pp |
4/10 E · 1/9 M · 1/1 H |
934.4 → 896.3 |
17.8 |
qwen38_27b |
Qwen3.8-27B-Weight-Space-Programming |
Dynamic 4S Mid+Late LoRA | 6/20 (30.0%) |
8/20 (40.0%) |
+10.0 pp |
6/10 E · 1/9 M · 1/1 H |
1013.6 → 1075.3 |
6.5 |
Fair-evaluation parity note:
qwen35_0_8bstops cleanly on[248046 (<|im_end|>), 248044 (<|endoftext|>)](eliminating multi-turn<tool_response>hallucination loops where pre-fix Base emitted2068.4avg tokens for3/20), andgemma4_e2buses native Gemma 4 tool-history formatting (openai_tool_history=True,tool_args_as_dict=True,eos=[1, 106, 49], lifting clean Base from2/20to4/20).
- Downloads last month
- 175
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "jstarks/gemma-4-E4B-it-weight-space-programming" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jstarks/gemma-4-E4B-it-weight-space-programming", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'