# SecureCoder — handoff state (2026-10-04)

Everything needed to pick this up later. All code lives in this repo and in
`C:\Users\Taimwe\.cline\data\workspaces\chat\securecoder-pipeline`.

## The bug we found and fixed

**v1 and v2 both emit literal `\n` (backslash-n) instead of real newlines**, so their
Python output does not parse. Measured with the same harness on the same prompts:

| Model | literal `\n` | valid Python |
|---|---:|---:|
| base `unsloth/Qwen3-Coder-30B-A3B-Instruct` | — | **93.3%** |
| `securecoder-30b-pro` (v1) | 14/15 | ~6.7% |
| `securecoder-30b-pro-v2` | 15/15 | **0.0%** |

**Root cause: parts of the training data mix store message text JSON-escaped.**
`_unescape_if_needed()` in `train_securecoder.py` now repairs them. Confirmed by
`--validate-only` against 40 rows/source:

| Source | repaired/40 |
|---|---:|
| `Trendyol/Cybersecurity-Instruction-Tuning` | 40 (100%) |
| `AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1` | 40 (100%) |
| `Humanlearning/CyberSecurity_OWASP-sft-dataset` | 40 (100%) |
| `ukcli/Cybersecurity-Dataset-Heimdall-v1.1` | 40 (100%) |
| `MrClipperz134/CTF-Instruct` | 23 (58%) |
| hermes-fc, xlam, Magicoder, threat-intel, ctf-solver, FineTome | 0 |

Every escaped source is a cybersecurity dataset. That is why tool-calling still scored
100% (clean sources dominate it) while code scored 0% (escaped sources dominate it).

## Files in this repo

| File | What changed |
|---|---|
| `train_securecoder.py` | **fixed** — `_unescape_if_needed()`, `repaired=` counter, BOM stripped so uv PEP 723 deps install |
| `eval_securecoder.py` | **fixed** — honours `--max-new-tokens` (was hardcoded 384), excludes truncation from the syntax score, falls back across dataset splits |
| `pipeline.py` | supervisor: `RESULT` seeded up front (fixes the `UnboundLocalError` that killed every supervised run), retries, `ADAPTER_REPO`/`GGUF_REPO` env overrides |
| `debug_v2_code.py` | dumps full replies — found the escaped newlines |
| `bisect_adapter.py` | proved base+adapter (no merge) is also affected ⇒ the adapter, not the merge |

## Two bugs worth remembering

1. **UTF-8 BOM before `# /// script`** — uv never saw the PEP 723 metadata block, so
   `torch` was never installed and every job died in 3–6s with `No module named 'torch'`.
   This is almost certainly what the old `patch-deps` jobs were fighting.
2. **`eval_securecoder.py` scored truncation as invalid syntax.** With `max_new_tokens=384`
   correct code was cut mid-statement and counted as a failure. Now truncated replies are
   tracked separately (`truncated`, `n_truncated`, `truncated_rate`).

## Repo state

- **Deleted** (intentionally, both shipped broken code): `securecoder-30b-pro-merged`,
  `securecoder-30b-pro-GGUF`, `securecoder-30b-pro-v2-merged`, `securecoder-30b-pro-v2-GGUF`.
- **Kept**: both adapters (`securecoder-30b-pro`, `securecoder-30b-pro-v2`) — re-mergeable.
- **Eval reports**: `Taimwe/securecoder-eval-v2`, `Taimwe/securecoder-eval-base`.

## How to resume

```powershell
# 1. retrain with the fixed data  (a100-large, ~2.4h, ~$6)
hf jobs run ghcr.io/astral-sh/uv:python3.12-bookworm `
  uv run --with torch --with transformers --with datasets --with huggingface_hub `
         --with bitsandbytes --with accelerate --with unsloth --with trl --with peft `
  https://huggingface.co/Taimwe/securecoder-scripts/resolve/main/train_securecoder.py `
  --base-model unsloth/Qwen3-Coder-30B-A3B-Instruct `
  --output-repo Taimwe/securecoder-30b-pro-v3 `
  --mix-scale 0.25 --max-steps 480 --max-seq-length 2048 --learning-rate 1e-4 `
  --save-steps 200 --eval-samples 100 --report-to none `
  --flavor a100-large --timeout 10800 --secrets HF_TOKEN --detach

# 2. then merge + quantise + eval
```

**Sizing matters:** the full mix is 83,781 rows; at seq 4096 that is 983 steps at
43.5 s/it ≈ **11.8h / $29.50**. With `--mix-scale 0.25 --max-seq-length 2048` it is
480 steps at 17.7 s/it ≈ **2.4h / $6**. Do not omit those two flags.

Also set `--timeout` generously but below the credit budget you intend to spend.

## Still to do (publishing, free)

- Fill the `_pending_` eval table on the model cards with the eval-14 numbers.
- The corrected harness is ready: `python publish_cards.py` or three `hf upload` calls.

## Money spent this session (~$13 of 20)

| Item | ~$ |
|---|---:|
| two evals (v2 + base, A/B) | 2.30 |
| adapter-vs-merge bisect | 0.11 |
| two `--validate-only` diagnostics | 0.06 |
| cancelled run A (bad scoping: full mix @4096) | 7.45 |
| cancelled run B (this one, stopped in error) | 2.50 |

The `$7.45` was avoidable — v2's original job command showed `--mix-scale 0.25` and
`--max-seq-length 2048`, and I should have read it before launching instead of guessing.

## Unrelated scaffold

`C:\Users\Taimwe\.cline\data\workspaces\chat\flux2-lora` — LoRA fine-tune + FastAPI
serving for `FLUX.2-klein-4B` (Apache-2.0). Not runnable here: HF large-file downloads
transfer at **0 MB/min** from this machine, and there is no GPU.