1.1.0 docs: architecture section, Jev 1.14.0 comparison, MTP wording

README.md

@@ -28,7 +28,7 @@ a choice, a calibrated confidence and the full distribution. Its [inference serv
 provides a `/v1/systemone` endpoint that follows the Jev request schema.
 
 [Inference server](https://github.com/riposta/CrossbowReviewer) · [Quickstart](docs/QUICKSTART.md) ·
-[Benchmarks](docs/BENCHMARKS.md) · [Artifact manifest](release-manifest.json) · 1.0 is kept under the `v1.0` tag.
+[Architecture](docs/ARCHITECTURE.md) · [Benchmarks](docs/BENCHMARKS.md) · [Artifact manifest](release-manifest.json) · 1.0 is kept under the `v1.0` tag.
 
 ## What's new in 1.1
 
@@ -69,6 +69,16 @@ one pass. See [docs/BENCHMARKS.md](docs/BENCHMARKS.md) for all tables.
 - **7 languages:** Java, Python, TypeScript, JavaScript, Go, C#, Rust.
 - **Full weights:** no adapter and no conversion step. Safetensors work with transformers; GGUF works with llama.cpp.
 
+## Architecture
+
+Qwen3.5-9B-Base (32 layers, 8 × (3 Gated DeltaNet + 1 Gated Attention), hidden 4096) with a rank-32 LoRA merged into
+the weights. No added layers or heads: all questions go in one prompt, the logits of the option letters are read at
+every `Answer:` slot in a single forward pass, and a fitted temperature turns them into calibrated probabilities.
+
+![Architecture](docs/assets/architecture.svg)
+
+Details, base-model table and training recipe: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
+
 ## Downloads
 
 | Files | Format and use | Size |
@@ -109,7 +119,8 @@ Pooled MCC (1 = perfect, 0 = chance), 95% cluster-bootstrap CI in [docs/BENCHMAR
 | **CrossbowReviewer 1.1** | 0.72 | 0.51 | 0.34 | **0.24** |
 | CrossbowReviewer 1.0 | 0.72 | 0.47 | 0.34 | 0.08 |
 | Qwen3.5-9B-Base (zero-shot) | 0.26 | −0.13 | 0.39 | 0.05 |
-| Jev (jev-1.13.0) | 0.57 | 0.25 | – | – |
+| Jev 1.14.0 (latest) | 0.58 | 0.25 | 0.34 | 0.10 |
+| Jev 1.13.0 | 0.57 | 0.25 | – | – |
 | basal-omni-1.0-max (zero-shot) | 0.60 | 0.26 | 0.50¹ | 0.14 |
 | Claude Opus 5.5 (reasoning) | 0.85 | 0.67 | 0.94¹ | – |
 | GPT-6 Sol (reasoning) | 0.80 | 0.84² | 0.77¹ | – |
@@ -162,8 +173,8 @@ path traversal), averaging both option orders, and LoRA on basal-omni (collapsed
 - **Optimization:** 3 rounds × 1 epoch, AdamW, lr 5e-5, bf16, 1× A100 (GCP). The best round was selected on a
   validation split.
 - **Calibration:** a single temperature (1.122) fitted on a validation split.
-- **Not included:** the multi-token-prediction head of the base model is not shipped (`mtp_num_hidden_layers: 0`),
-  because decisions never generate text.
+- **Not included:** the multi-token-prediction head of the base model. No MTP weights are in this repo (`config.json`
+  still carries the base value `mtp_num_hidden_layers: 1`), because decisions never generate text.
 
 ## License
 

SHA256SUMS

@@ -1,13 +1,15 @@
 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  LICENSE
 930472a7a4b21e39db2a68aafcb1cc09984a4c7f88e6072682eef75b2ec844b2  NOTICE
-cdbd79ef17de41d48d3e5daba3fde2fd3e7cb1cea209de87b47fa73496376fab  README.md
+90035aa0d8ec551faf8ea3a1c6857de18732c466e1edcce77e506dbed7320530  README.md
 a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715  chat_template.jinja
 4a9248516d271480884c80271eb8023757bd313932b07facd4cd6a820737ae83  config.json
 ba9d608ecc284b14389deb69e3266ef91e3790a5c76593149e9103c9c25ef779  decision_config.json
-3dfacdbcd5bca6ee74ac3bd058936067a68857982d65c280f19e9f1f2fd9c013  docs/BENCHMARKS.md
+42f7e41a7aab678634e43e0f35edc1cd3cc9a19d59635f411809093ff92d13d7  docs/ARCHITECTURE.md
+b81a61d000f795b48a583ef2c3b01e4eb8ec1d8e94ad1284254a0bf2719e9825  docs/BENCHMARKS.md
 1f8496c6b8ce1e69660df2bcb09bc991150c42622551a2e2c09df974a1ac151c  docs/QUICKSTART.md
+9f3e8aa4db68dd26d3659091e9e2a065116ea8aff18b95ebd2402cf872586220  docs/assets/architecture.svg
 8528e3180091f4c1d2c321300f89a21149572e7fe3fe74c1342679eb3c3dacb5  docs/assets/release_mcc.png
-01fac4f52eafc873c80db7394583d53ed23fae8620d753337f0365d11ee26d96  docs/benchmarks.json
+fa7f5e52138601661415c58cd95a3cd654b2919d2af31060df2b9e8f0a78e567  docs/benchmarks.json
 62153eb6c69f2e1f426beaa8002b7186437e949c7588167085df14e10e9c0a73  generation_config.json
 3d61d66a0100ac3a1d09291ec1de1155579941b6785b049736db54bca91fe126  gguf/CrossbowReviewer-9B-Q8_0.gguf
 c433715d8ec5e6497aabf0455175752e6f3698c22db6cd4b84853c9038cd3f7c  model-00001-of-00004.safetensors

docs/ARCHITECTURE.md

@@ -0,0 +1,60 @@
+# Architecture
+
+CrossbowReviewer is [Qwen3.5-9B-Base](https://huggingface.co/Qwen/Qwen3.5-9B-Base) with a LoRA merged into the
+weights. The architecture is unchanged: no added layers and no classification head. What differs is how the model is
+trained and read out: **slot decoding**.
+
+![Architecture](assets/architecture.svg)
+
+## Base model
+
+| | |
+|---|---|
+| Type | causal language model (`Qwen3_5ForCausalLM`, text part only) |
+| Parameters | 9B dense (LoRA is merged, so this file set has no adapter) |
+| Layers / hidden size | 32 / 4096, FFN 12288 |
+| Layout | 8 × (3 × Gated DeltaNet → 1 × Gated Attention) |
+| Gated DeltaNet | 32 value heads, 16 QK heads, head dim 128, conv kernel 4 |
+| Gated Attention | 16 query heads, 4 KV heads (GQA), head dim 256, partial RoPE (0.25) |
+| Vocabulary | 248 320 |
+| Context | 262 144 tokens natively |
+| MTP head | **not shipped.** `config.json` still carries the base value `mtp_num_hidden_layers: 1`, but no MTP weights are in the repo and decisions never generate text |
+
+Three of every four layers are linear attention, so prefill cost grows gently with file length. This is what makes a
+single pass over a whole file and ~107 rules cheap.
+
+## Slot decoding
+
+1. **Prompt.** Header, then `state` (a JSON object with the language and the code), then every question as
+   `### name`, instructions, lettered options and `Answer:`.
+2. **One forward pass.** The answers are never in the input, so a slot cannot see the answers to earlier slots.
+   Training and inference see the same sequence, and all questions share one prefill.
+3. **Read-out.** The LM head runs only at the positions of the `Answer:` slots, not over the whole sequence. For each
+   slot the logits of the option letters (` A`, ` B`, up to 26) are taken.
+4. **Calibration.** `softmax(logits / T)` with a single temperature `T = 1.122` (`decision_config.json`). The server
+   can add a per-category scale and accept/defer thresholds. For data-flow security no threshold could be certified.
+5. **Typed output.** `noul`: yes/no (`no` means the rule is violated); `choice`: up to 26 options; `score`: 2–10
+   levels. Every answer has a `choice`, a `confidence` and the full `distribution`.
+
+## Training
+
+- **Method:** LoRA on all linear modules (`q/k/v/o`, `gate/up/down` and the DeltaNet projections `in_proj_qkv`,
+  `in_proj_z`, `in_proj_a`, `in_proj_b`, `out_proj`). Rank 32, alpha 64, dropout 0.05.
+- **Loss:** cross-entropy over the option-letter logits at each slot. Each sample shows a random subset of the questions,
+  always including its target rule.
+- **Data:** 4 866 samples: synthetic snippets, 445 vulnerable/fixed pairs with a verified contrast and 900 real-code files
+  with soft labels. 3 rounds × 1 epoch, lr 5e-5, bf16, 1× A100.
+
+## What is not new
+
+Unlike recent architecture releases, such as sparse attention, gated residuals or n-gram embeddings in
+[Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next), CrossbowReviewer does not change the network. The
+gain comes from data, the slot read-out and calibration, so any improvement in the base model can be reused by
+retraining the same recipe.
+
+## Variants we tried
+
+All documented in the
+[research log](https://github.com/riposta/CrossbowReviewer-Pipeline/blob/main/docs/RESEARCH_UPGRADE.md): a prefix layout
+with a fixed rule list before the code (worse on OWASP injection and path traversal), contrast-format training (worse
+on real code), averaging both option orders (no gain) and LoRA on basal-omni (collapsed to chance).

docs/BENCHMARKS.md

@@ -34,7 +34,8 @@ Pooled MCC with 95% cluster-bootstrap CIs:
 | **CrossbowReviewer 1.1** | 0.72 (0.69–0.75) | 0.955 | 0.892 | 0.69 | 0.97 | 0.83 | 0.010 |
 | CrossbowReviewer 1.0 | 0.72 (0.68–0.75) | 0.959 | 0.905 | 0.65 | 0.98 | 0.82 | 0.009 |
 | Qwen3.5-9B-Base (zero-shot) | 0.26 (0.21–0.30) | 0.653 | 0.463 | 0.36 | 0.87 | 0.62 | 0.040 |
-| Jev | 0.57 (0.54–0.61) | 0.883 | 0.684 | 0.81 | 0.82 | 0.81 | 0.046 |
+| Jev 1.14.0 (latest) | 0.58 (0.55–0.62) | 0.891 | 0.706 | 0.81 | 0.83 | 0.82 | 0.054 |
+| Jev 1.13.0 | 0.57 (0.54–0.61) | 0.883 | 0.684 | 0.81 | 0.82 | 0.81 | 0.046 |
 | basal-omni-1.0-max (zero-shot) | 0.60 (0.56–0.64) | 0.896 | 0.771 | 0.71 | 0.89 | 0.80 | 0.076 |
 | Claude Opus 5.5 | 0.85 (0.82–0.87) | 0.985 | 0.960 | 0.86 | 0.97 | 0.92 | 0.120 |
 | GPT-6 Sol | 0.80 (0.78–0.83) | 0.976 | 0.930 | 0.95 | 0.91 | 0.93 | 0.021 |
@@ -47,7 +48,8 @@ likely inflated)
 | **CrossbowReviewer 1.1** | 0.51 (0.46–0.54) | 0.961 | 0.705 | 0.36 | 0.99 | 0.67 | 0.014 |
 | CrossbowReviewer 1.0 | 0.47 (0.42–0.53) | 0.938 | 0.576 | 0.35 | 0.99 | 0.67 | 0.006 |
 | Qwen3.5-9B-Base (zero-shot) | -0.13 (-0.16–-0.10) | 0.309 | 0.046 | 0.04 | 0.75 | 0.39 | 0.029 |
-| Jev | 0.25 (0.21–0.28) | 0.799 | 0.174 | 0.67 | 0.76 | 0.72 | 0.030 |
+| Jev 1.14.0 (latest) | 0.25 (0.21–0.28) | 0.801 | 0.178 | 0.68 | 0.75 | 0.72 | 0.031 |
+| Jev 1.13.0 | 0.25 (0.21–0.28) | 0.799 | 0.174 | 0.67 | 0.76 | 0.72 | 0.030 |
 | basal-omni-1.0-max (zero-shot) | 0.26 (0.22–0.31) | 0.802 | 0.224 | 0.54 | 0.85 | 0.70 | 0.066 |
 | Claude Opus 5.5 | 0.67 (0.63–0.72) | 0.990 | 0.905 | 0.49 | 1.00 | 0.74 | 0.086 |
 | GPT-6 Sol | 0.84 (0.81–0.87) | 0.981 | 0.886 | 0.83 | 0.99 | 0.91 | 0.037 |
@@ -59,7 +61,8 @@ likely inflated)
 | **CrossbowReviewer 1.1** | 0.30 (0.22–0.40) | 0.697 | 0.715 | 0.74 | 0.55 | 0.65 | 0.168 |
 | CrossbowReviewer 1.0 | 0.28 (0.20–0.36) | 0.715 | 0.722 | 0.94 | 0.26 | 0.60 | 0.239 |
 | Qwen3.5-9B-Base (zero-shot) | 0.37 (0.29–0.45) | 0.727 | 0.713 | 0.93 | 0.37 | 0.65 | 0.025 |
-| Jev | 0.30 (0.22–0.37) | 0.719 | 0.649 | 0.97 | 0.24 | 0.60 | 0.209 |
+| Jev 1.14.0 (latest) | 0.33 (0.25–0.40) | 0.725 | 0.687 | 0.95 | 0.30 | 0.62 | 0.178 |
+| Jev 1.13.0 | 0.30 (0.22–0.37) | 0.719 | 0.649 | 0.97 | 0.24 | 0.60 | 0.209 |
 | basal-omni-1.0-max (zero-shot) | 0.50 (0.45–0.56) | 0.825 | 0.800 | 1.00 | 0.40 | 0.70 | 0.257 |
 | Claude Opus 5.5 | 0.94 (0.90–0.97) | 0.983 | 0.977 | 1.00 | 0.94 | 0.97 | 0.059 |
 | GPT-6 Sol | 0.77 (0.72–0.82) | 0.935 | 0.905 | 1.00 | 0.75 | 0.88 | 0.106 |
@@ -71,6 +74,7 @@ likely inflated)
 | **CrossbowReviewer 1.1** | 0.34 (0.23–0.42) | 0.722 | 0.700 | 0.88 | 0.42 | 0.65 | 0.150 |
 | CrossbowReviewer 1.0 | 0.34 (0.25–0.41) | 0.738 | 0.750 | 0.94 | 0.32 | 0.63 | 0.238 |
 | Qwen3.5-9B-Base (zero-shot) | 0.39 (0.31–0.45) | 0.744 | 0.733 | 0.93 | 0.41 | 0.67 | 0.027 |
+| Jev 1.14.0 (latest) | 0.34 (0.25–0.41) | 0.763 | 0.719 | 0.94 | 0.32 | 0.63 | 0.189 |
 
 **Vulnerability/fix pairs** (98 pairs = 196 decisions)
 
@@ -79,6 +83,7 @@ likely inflated)
 | **CrossbowReviewer 1.1** | 0.24 (0.14–0.34) | 0.632 | 0.39 | 0.83 | **0.235** | 0.197 |
 | CrossbowReviewer 1.0 | 0.08 (-0.03–0.19) | 0.506 | 0.05 | 0.98 | 0.041 | 0.353 |
 | Qwen3.5-9B-Base (zero-shot) | 0.05 (0.00–0.14) | 0.529 | 0.05 | 0.97 | – | 0.247 |
+| Jev 1.14.0 (latest) | 0.10 (0.03–0.17) | 0.546 | 0.50 | 0.60 | 0.112 | 0.220 |
 | basal-omni-1.0-max (zero-shot) | 0.14 (0.05–0.21) | 0.574 | 0.47 | 0.66 | 0.163 | 0.319 |
 
 Pair accuracy = both versions of a pair judged correctly.
@@ -105,6 +110,15 @@ Pair accuracy = both versions of a pair judged correctly.
 - **Security stays a human/reasoning-model decision.** 1.1 recognizes 42–55% of the safe OWASP traps (1.0: 26–32%),
   better but still far from Opus 5.5 (94%).
 
+## Against the latest Jev
+
+We re-ran everything with **jev-1.14.0** (the current `jev-latest`); 1.0's card used jev-1.13.0, which is kept in the
+tables for reference. 1.14.0 is only marginally better than 1.13.0 on every set. CrossbowReviewer 1.1 stays well ahead
+of it on real code (MCC 0.51 vs 0.25, AUC 0.961 vs 0.801, AP 0.71 vs 0.18) and on vulnerability/fix pairs (pair
+accuracy 0.235 vs 0.112) and on synthetic data (MCC 0.72 vs 0.58). Jev flags more violations (recall 0.81 vs 0.69
+on synthetic, 0.68 vs 0.36 on real code) but pays with 25% false positives on real code versus 1%. On OWASP the two
+are statistically indistinguishable (held-out MCC 0.34 vs 0.34).
+
 ## Other benchmarks considered
 
 We surveyed public benchmarks that could extend this evaluation. None is included in the numbers above.

docs/benchmarks.json

@@ -58,7 +58,26 @@
      0.689107245171945
     ]
    },
-   "Jev": {
+   "Jev 1.14.0": {
+    "n": 2838,
+    "pos": 691,
+    "mcc": 0.5834351144821561,
+    "auc": 0.8905361838313751,
+    "ap": 0.7064306670126849,
+    "recall": 0.8133140376266281,
+    "specificity": 0.8258034466697718,
+    "bal_acc": 0.8195587421482,
+    "ece": 0.05366102889358701,
+    "mcc_ci": [
+     0.5482742738958795,
+     0.6178649042752993
+    ],
+    "auc_ci": [
+     0.8747442690257708,
+     0.9031681062486749
+    ]
+   },
+   "Jev 1.13.0": {
     "n": 2838,
     "pos": 691,
     "mcc": 0.5735505633672757,
@@ -193,7 +212,26 @@
      null
     ]
    },
-   "Jev": {
+   "Jev 1.14.0": {
+    "n": 5361,
+    "pos": 370,
+    "mcc": 0.2453543318204322,
+    "auc": 0.8010554132573768,
+    "ap": 0.17784154782117428,
+    "recall": 0.6810810810810811,
+    "specificity": 0.7523542376277299,
+    "bal_acc": 0.7167176593544056,
+    "ece": 0.03080768513337065,
+    "mcc_ci": [
+     0.20586233752208488,
+     0.2758830661952689
+    ],
+    "auc_ci": [
+     null,
+     null
+    ]
+   },
+   "Jev 1.13.0": {
     "n": 5361,
     "pos": 370,
     "mcc": 0.24813049944531618,
@@ -328,7 +366,26 @@
      0.7772480718704975
     ]
    },
-   "Jev": {
+   "Jev 1.14.0": {
+    "n": 435,
+    "pos": 215,
+    "mcc": 0.32949472283317666,
+    "auc": 0.7251162790697674,
+    "ap": 0.6869969579373971,
+    "recall": 0.9534883720930233,
+    "specificity": 0.29545454545454547,
+    "bal_acc": 0.6244714587737844,
+    "ece": 0.17781609195402298,
+    "mcc_ci": [
+     0.2543787415657978,
+     0.4035649670100956
+    ],
+    "auc_ci": [
+     0.6782645502645502,
+     0.7726313770617568
+    ]
+   },
+   "Jev 1.13.0": {
     "n": 435,
     "pos": 215,
     "mcc": 0.2975956201756454,
@@ -462,6 +519,25 @@
      0.7048918279758255,
      0.7927573203194321
     ]
+   },
+   "Jev 1.14.0": {
+    "n": 429,
+    "pos": 212,
+    "mcc": 0.338197199741519,
+    "auc": 0.7631618989653074,
+    "ap": 0.7190012552507654,
+    "recall": 0.9433962264150944,
+    "specificity": 0.3225806451612903,
+    "bal_acc": 0.6329884357881923,
+    "ece": 0.18899766899766893,
+    "mcc_ci": [
+     0.24606370318886614,
+     0.40690936752428153
+    ],
+    "auc_ci": [
+     0.7238283627510651,
+     0.8062861018729214
+    ]
    }
   },
   "pairs": {
@@ -522,6 +598,25 @@
      0.5582569762598917
     ]
    },
+   "Jev 1.14.0": {
+    "n": 196,
+    "pos": 98,
+    "mcc": 0.10257624248277729,
+    "auc": 0.546438983756768,
+    "ap": 0.5555946836193321,
+    "recall": 0.5,
+    "specificity": 0.6020408163265306,
+    "bal_acc": 0.5510204081632653,
+    "ece": 0.2204081632653061,
+    "mcc_ci": [
+     0.03184469354605652,
+     0.16754156331667822
+    ],
+    "auc_ci": [
+     0.5219179508538109,
+     0.5741357767596834
+    ]
+   },
    "basal-omni-1.0-max": {
     "n": 196,
     "pos": 98,

release-manifest.json

@@ -28,8 +28,8 @@
     },
     {
       "path": "README.md",
-      "bytes": 9314,
-      "sha256": "cdbd79ef17de41d48d3e5daba3fde2fd3e7cb1cea209de87b47fa73496376fab"
+      "bytes": 9968,
+      "sha256": "90035aa0d8ec551faf8ea3a1c6857de18732c466e1edcce77e506dbed7320530"
     },
     {
       "path": "chat_template.jinja",
@@ -46,16 +46,26 @@
       "bytes": 618,
       "sha256": "ba9d608ecc284b14389deb69e3266ef91e3790a5c76593149e9103c9c25ef779"
     },
+    {
+      "path": "docs/ARCHITECTURE.md",
+      "bytes": 3531,
+      "sha256": "42f7e41a7aab678634e43e0f35edc1cd3cc9a19d59635f411809093ff92d13d7"
+    },
     {
       "path": "docs/BENCHMARKS.md",
-      "bytes": 8799,
-      "sha256": "3dfacdbcd5bca6ee74ac3bd058936067a68857982d65c280f19e9f1f2fd9c013"
+      "bytes": 9935,
+      "sha256": "b81a61d000f795b48a583ef2c3b01e4eb8ec1d8e94ad1284254a0bf2719e9825"
     },
     {
       "path": "docs/QUICKSTART.md",
       "bytes": 3131,
       "sha256": "1f8496c6b8ce1e69660df2bcb09bc991150c42622551a2e2c09df974a1ac151c"
     },
+    {
+      "path": "docs/assets/architecture.svg",
+      "bytes": 5944,
+      "sha256": "9f3e8aa4db68dd26d3659091e9e2a065116ea8aff18b95ebd2402cf872586220"
+    },
     {
       "path": "docs/assets/release_mcc.png",
       "bytes": 80837,
@@ -63,8 +73,8 @@
     },
     {
       "path": "docs/benchmarks.json",
-      "bytes": 14054,
-      "sha256": "01fac4f52eafc873c80db7394583d53ed23fae8620d753337f0365d11ee26d96"
+      "bytes": 16186,
+      "sha256": "fa7f5e52138601661415c58cd95a3cd654b2919d2af31060df2b9e8f0a78e567"
     },
     {
       "path": "generation_config.json",
@@ -111,5 +121,6 @@
       "bytes": 1128,
       "sha256": "8671bed7c852ce9e661be94f179a7b4ffd091c2a65aea0363e5501c20318ee45"
     }
-  ]
+  ],
+  "notes": "1.1.0 docs update: architecture section, Jev 1.14.0 comparison, MTP wording"
 }