# Progress

**Current phase:** Phase 1 (Foundation) complete + **pitch-ready POC, now actually deployed to Hugging Face Spaces (two live Spaces).** Experiment 3's adapter (`models/exp3_gemma3_4b_public_v0_1`) is thoroughly evaluated (two independent large-n draws agree: headline generation ROUGE-L 0.157→~0.37 real win, translation en→te 40.2→~35.4 real modest dip, te→en 36.5→~40.7 real modest improvement, sentence_compression ~0.25→~0.22 small dip, GEC 88.8→87.8 flat). A real FastAPI + live-wired-UI product exists (`server/`, `ui/rachana-desk.html`) covering headline/grammar/paraphrase/translate/summarize (sentence-level)/transliterate (now with real-time in-canvas suggestions, see below) plus a working Research Assistant (controlled RAG over 4 verified live Telugu news RSS feeds, two-step summarization, real citations), a reporter→editor review-queue workflow (submit/approve/send-back), all committed, pushed, and **verified end-to-end against the live production Space via real Playwright browser sessions** (not just API calls or code review).

**Deployment state (2026-07-23):** repo is git-initialized (see `decisions.md`). **`rachana-news`** (Docker SDK, the full product) is running on paid `t4-small` GPU and **the user switched it from private to public this session** - this incidentally fixed a real blocker: private-Space browser navigation (as opposed to plain API calls with a Bearer token) was hitting an HF auth/routing quirk that 404'd every Playwright page load, even though direct HTTP calls with the same token worked fine. Now public, browser automation against it just works. `rachana-news-demo` (Gradio SDK, ZeroGPU) unchanged this session. Everything built 2026-07-22/23 (the field-report fixes, the review-queue POC, the research-matching fix, transliteration wired into Compose, and the new real-time suggestion feature) is **pushed live and confirmed working via a real Playwright browser audit against production** - screenshots, zero console errors, exact expected output at every step (see `decisions.md` for the full list).
**Last session:** 2026-07-23.

## Done

- [x] PRD drafted (`Rachana_News_PRD.md`, now v0.2)
- [x] Obsidian vault set up for this repo (`.obsidian/`, registered in Obsidian's vault list, `Home.md` entry point)
- [x] Baseline eval harness built: `scripts/prepare_eval_data.py` → `run_eval.py` → `score_eval.py`
- [x] Baseline eval run on unmodified `gemma3:4b` across 8 capabilities (n=5-10 each) — results in `reports/baseline_v0.md` / `data/processed/automated_scores.json`
- [x] PRD §11 open question #1 answered: continued pretraining likely NOT needed; real gaps are headline coherence, summarization register, transliteration metric caveats
- [x] `README.md` written (setup + pipeline usage)
- [x] Non-technical status report built for cofounders — `reports/cofounder-status-2026-07-13.html`
- [x] Gemma 3 license terms verified: commercial use + fine-tuning permitted under Google's custom ToS (Prohibited Use Policy must pass through to downstream customers). See `decisions.md` and PRD §9.
- [x] Gemma 3 vs. Gemma 4 settled: staying on Gemma 3 4B IT + QLoRA for v1; Gemma 4 (real, released 2026-07-16) is an eval-only comparison candidate later.
- [x] `gemma3:4b` tokenizer probed via Ollama's local API (no HF token needed). seq_len 640 confirmed sufficient. `scripts/probe_tokenizer.py`, `reports/tokenizer_probe.md`.
- [x] Real dataset audit done for all 6 v0.1 public-data capabilities. Two findings changed the plan: **GEC has only 699 real pairs** (needs synthetic supplement), **public "summarization" data is sentence-level compression, not article summarization** (relabeled `sentence_compression`). Full counts in `data-sources.md`.
- [x] v0.1/v0.2 staging clarified: v0.1 is a disposable pipeline-validation checkpoint; v0.2 restarts from base on a blended mixture. See `decisions.md`.
- [x] Notes→article / press-release→news pipeline **designed** (two-stage decomposition + TeClass reverse-generation technique) — not yet built/piloted.
- [x] v0.1 public-data SFT pipeline built (`scripts/prepare_sft_data.py`) — `sft_train_v0_1.jsonl` (26,735 rows, 7 tasks incl. transliteration) + `sft_eval_protected_v0_1.jsonl` (1,100 rows), zero train/eval overlap verified.
- [x] Native-Windows training stack installed and verified end-to-end (torch/transformers/peft/bitsandbytes/trl/accelerate/wandb). Real bug hit + fixed: RTX 5050 is Blackwell/sm_120, needed `cu128` torch build not `cu124` (see `errors.md`). `HF_TOKEN`/`WANDB_API_KEY` wired via `.env` + `python-dotenv`.
- [x] **Experiment 1 (Gemma 3 1B pipeline check) — passed.** 980 examples/7 tasks, 1 epoch. Loss 5.38→3.1, adapter save/reload/generation-diff all confirmed working. Base model's meta-commentary-rambling failure mode visibly suppressed post-adapter. `scripts/train_qlora.py`, `scripts/compare_base_vs_adapter.py`.
- [x] Comprehensive wandb logging added to `train_qlora.py`: per-task `gen_eval/*` metrics (reusing `scripts/metrics.py`, shared with `score_eval.py`), sample-generation `wandb.Table`, GPU memory/throughput (`resource/*`), full config logging, dedicated `rachana-news` project with explicit run names/tags. One real bug found+fixed (wandb step-race silently dropping a log call).
- [x] Roman-Telugu comprehension probe: Gemma 3 4B reads Roman/mixed-script Telugu well (~90% after manual review, n=20) — no dedicated training needed. Two real failure modes logged (script derailment on one spelling variant; double-counting reasoning error on redundant bilingual phrasing — relevant to the future notes→article grounding-check design). Confirms transliteration *production* stays a separate IndicXlit module and doesn't need main-model training either direction. `scripts/prepare_roman_telugu_eval.py`.
- [x] Architecture decision: transliteration dropped entirely from the main model's SFT scope (IndicXlit module handles production; comprehension confirmed fine zero-shot). Applied starting with Experiment 2's data.
- [x] **Experiment 2 (Gemma 3 4B, small test) — passed.** 4,335 examples (6 tasks, no transliteration), 1 epoch. **VRAM peaked at 7.17GB/8GB** — tight, no headroom for seq_len/batch/target-module increases. **9.7 sec/step measured** → Experiment 3 estimated ~8 hours. Quality clearly improved over Experiment 1 (mean_token_accuracy ~0.70 vs 0.51; GEC GLEU 88.8, translation chrF 40.2/36.5). **Headline generation is the weakest capability (ROUGE-L 0.157)** — flagged for specific attention. Full numbers pulled via wandb API (run page itself needs browser login, not fetchable directly) — see `decisions.md`.
- [x] `google/translategemma-4b-it` tested as a translation teacher-model candidate — **inconclusive** (chrF roughly even/slightly worse than baseline Gemma 3 4B on the same 10 examples, but references are known-noisy mined data and n=5/direction is too thin to trust either way). PRD open question #3 not resolved by this test. `scripts/test_translategemma.py`.
- [x] GEC corrupted-reference generator built and run at **50% scope** (explicit instruction): 874 synthetic correction pairs + 291 synthetic identity pairs generated (`scripts/generate_gec_synthetic.py`, few-shot primed from real BHASHA error examples, via Ollama). Smoke-test caught and fixed one real quality issue (word-diff-count filter, not a length-ratio filter — see `decisions.md`). **At-scale spot-check (15 examples) found ~1/3 still have quality issues** even after filtering: some backwards-labeled (model "corrected" a pre-existing typo in the mined IndicParaphrase source rather than introducing a new error), some are meaning-changes rather than realistic grammar mistakes. A second automated filter pass was proposed and **explicitly deferred by the user** — this batch is NOT yet merged into any training run.
- [x] Experiment 3 data built (`scripts/prepare_experiment3_data.py`): 23,735 train / 900 eval rows — full v0.1 mixture minus transliteration, **deliberately excluding** the unfiltered GEC synthetic batch above (GEC stays at the same 735 real+synthetic-identity examples Experiment 2 already validated). Data files (`exp3_train.jsonl`/`exp3_eval.jsonl`) are valid and ready for a future run.
- [x] Experiment 3 training was started, then **stopped mid-run by explicit user request** before completing epoch 1 — no checkpoint was saved (`save_strategy="epoch"` only checkpoints at epoch boundaries), so resuming means starting over, not resuming partial progress. See `decisions.md`.
- [x] Accelerate/Unsloth/PyTorch Lightning all resolved — **none adopted, Experiment 3 ran on the proven native stack.** Accelerate: already in use transparently, no action. PyTorch Lightning: rejected without testing (training-loop framework, not a speed/VRAM optimizer — no benefit over TRL's `SFTTrainer`, which already sits on HF `Trainer`/Accelerate). Unsloth: actually validated in an isolated `.venv-unsloth` (dry-run showed it'd downgrade the main venv's proven torch build, so isolated first). Real measured results: VRAM peak genuinely lower (4B: 5.20GB vs. proven stack's 7.17GB, -27%; 1B: 1.68GB vs. 3.22GB, -48%) but **speed is worse, not better** — 4B: ~23.3-23.8 sec/step vs. proven stack's measured 9.7 sec/step (~2.4x slower); 1B: roughly parity. Root cause: no Flash-Attention-2 build available for this native-Windows + Blackwell (sm_120) + torch 2.10/cu128 combo (Unsloth's own banner shows `FA2 = False`, falls back to slower Xformers). Full numbers, two real bugs hit+fixed (import order, missing `formatting_func`), and the reasoning are in `decisions.md`/`errors.md` (2026-07-19 entries). `.venv-unsloth` and `scripts/train_unsloth_validate.py` kept in place for a cheap recheck later if/when FA2 gets native-Windows Blackwell support.
- [x] Follow-up question on WSL2 + real Flash-Attention-2: reasoned (not tested) estimate was modest speedup at best (~10-20%, not 2x) given seq_len 640 is short enough that attention is a small slice of a 4B model's compute — logged, not acted on, since Experiment 3 finished on the existing stack before this became relevant to test.
- [x] **Experiment 3 (Gemma 3 4B, full v0.1 run) — completed.** 23,735 examples/6 tasks, 2 epochs, 9h23m wall-clock, no errors, adapter saved to `models/exp3_gemma3_4b_public_v0_1`. Peak VRAM 7.167GB (matches Experiment 2's 7.17GB almost exactly — confirms peak VRAM tracks batch/seq_len/model size, not dataset size, now checked twice). **Mixed per-task results vs. Experiment 2** — headline_generation ROUGE-L 0.157→0.460 (big win, was the flagged-weakest capability), paraphrase ROUGE-L ~0.25→0.413 (win), but sentence_compression ROUGE-L ~0.25→0.190, GEC GLEU 88.8→83.7, and translation en→te chrF 40.2→23.98 all dipped, while translation te→en chrF jumped 36.5→56.5. **Do not trust the dips/swings yet** — training's `gen_eval` callback only samples 3 rows/task, same too-small-n problem already flagged for the baseline eval; the opposite-direction en→te vs. te→en swing especially smells like sampling noise rather than a real trade-off. Full numbers and reasoning in `decisions.md`.
- [x] Rachana News writing-assistant UI/UX — mockup built, published as a Claude artifact (not part of the training repo), then redesigned per explicit feedback from "vintage press desk" to a modern-professional visual system (clean cool-white/navy palette, elevated shadow cards instead of dashed/flat borders, sans-only bold typography, status dots instead of accent-rail stripes on history). Clickable prototype covering all 6 core capabilities, with the PRD's fact-preservation/grounding safeguard surfaced as a first-class UI element. URL: https://claude.ai/code/artifact/b57807ab-7930-4715-95df-d02d8ab14cd7 — further feedback still welcome.
- [x] **Proper-scale eval of Experiment 3's adapter — done, and it changes the story.** Built `scripts/eval_adapter.py` (loads base+adapter via PEFT, generates over a real eval file, writes `score_eval.py`-compatible output) and ran it on the full protected eval set (n=100-200/task, `data/processed/exp3_eval.jsonl`). **The training callback's n=3 numbers were mostly noise, not a real signal**: headline_generation ROUGE-L 0.460(callback)→0.388(proper) — still a big real win vs. Experiment 2's 0.157. translation en→te chrF 23.98(callback)→35.52(proper) vs. Experiment 2's 40.2 — a modest real dip, not a crash. translation te→en chrF 56.54(callback)→39.92(proper) vs. Experiment 2's 36.5 — a modest real improvement, not a doubling. GEC GLEU 83.71(callback)→87.84(proper) vs. Experiment 2's 88.8 — essentially flat, no real regression. sentence_compression ROUGE-L 0.190(callback)→0.222(proper) vs. ~0.25 — essentially flat. Full numbers/reasoning in `decisions.md`.
- [x] Second independent eval sample built and scored (`scripts/prepare_eval_resample2.py`) — 2000 fresh rows (400/task) for headline_generation/sentence_compression/translation_en_to_te/translation_te_to_en, drawn from genuine spare data pool, explicitly excluded against every row already used in train+eval, independent shuffle seed. **GEC excluded by necessity** — all 699 real BHASHA pairs already fully spent, zero spare, a permanent data-scarcity limit not an oversight. **Result: the two independent draws agree closely on every task** (e.g. headline ROUGE-L 0.388 vs. 0.353, translation en_to_te chrF 35.52 vs. 35.30) — confirms v0.1's per-task quality profile is real and settled, not sampling noise. Full comparison table in `decisions.md`.
- [x] **Pre-launch pitch-readiness checks — real evidence gathered, not assumptions.** (1) Headline hallucination spot-check (20 real generations): 19/20 accurate, 1 real failure — a headline asserted a gang had been "caught" when the source only said police intended to catch them soon (a completed-vs-pending-action fabrication). (2) Zero-shot full-article summarization (6 real TeClass articles, single-step prompt): confirmed this was never a trained/validated capability at any point in the project (even the original baseline eval's "summarization" template was sentence-level) — 2/6 worked well, 3/6 defaulted to copying the article's first sentence, 1/6 just continued/copied the source. (3) Roman-Telugu demo-input robustness (6 realistic romanized reporter-note inputs): found 3 real problems — a headline mis-framed scheduled maintenance as an emergency ("burst" vs. "repair"), GEC on romanized input dropped a key fact ("రైతులు"/farmers) entirely, and a spelling-variant stress test produced a garbled word ("కోర్ల" instead of "కోట్లు"). All three concentrate in one root cause: the LLM handling raw Roman-script input directly. Full results: `reports/headline_hallucination_sample.jsonl`, `reports/prelaunch_checks_results.json`.
- [x] IndicXlit transliteration module — investigated as a possible input-normalization guardrail, **not usable today**. Official `ai4bharat-transliteration` package depends on `fairseq`, whose pip packaging is currently broken (real build failure reproduced, not hypothetical). A community fairseq-free port (`psidharth567/indic-xlit-270M`, real HF safetensors model, custom code reviewed and found safe to run) was tested directly and produced garbage output (wrong-script symbols) regardless of input — either broken or missing undocumented usage details not published in its own README. **Decision: no automatic Roman→Telugu normalization for this POC** — shipped an honest live UI notice instead ("type/paste Telugu script for best results") when Roman/mixed-script input is detected, rather than silently mishandling it.
- [x] **Built and verified a real, live, pitch-ready POC** (not a mockup) — full plan in `~/.claude/plans/polished-snacking-frog.md`. New: `server/app.py` (FastAPI, loads base model + Experiment 3 adapter once at startup, same pattern as `scripts/eval_adapter.py`), `server/guardrails.py` (script detection + a word-boundary-based "completed-action" hallucination check — see real bug found+fixed below), `server/research.py` (controlled RAG). `ui/rachana-desk.html` rewired from canned demo data to real `fetch()` calls against the backend; Accept/Edit/Flag now POST to `/api/feedback` (`data/processed/usage_feedback.jsonl`) — the flywheel data-capture mechanism is live from the first real use, not a future add-on. All endpoints tested with real HTTP calls against the live model, not just code review.
  - **Real bug found and fixed**: the first version of the completed-action guardrail matched only full verb forms (e.g. "పట్టుకున్నారు") to avoid false-positiving on future-tense forms sharing a prefix (e.g. "పట్టుకుంటామని") — but real Telugu headlines commonly use the terse standalone form "పట్టు" alone, which the full-form-only check completely missed on the exact real hallucination case from the spot-check above. Fixed by switching to whitespace-tokenized word-boundary matching (exact token equality, not substring) — this correctly catches the terse form while still not false-positiving on the fused future-tense word. Verified against the real failing case (now flags correctly) and three clean cases (no false positives).
  - **Research Assistant is real and working**: verified 4 live Telugu news RSS feeds (BBC Telugu, Sakshi, TV9 Telugu, NTV Telugu) — 2 other candidates (Eenadu, Andhra Jyothy) were tried and dropped because they returned no entries/404, not assumed to work. A two-step summarization prompt (extract key facts, then combine) was used specifically to mitigate the first-sentence-extraction bug found in zero-shot summarization testing — verified with real queries ("డెంగ్యూ వ్యాక్సిన్", "రోహిత్ శర్మ క్రికెట్") against the live feeds: correct retrieval, real citations, and summaries that actually synthesize rather than copy the first sentence. No-match queries correctly return an empty, honest result rather than a fabricated one.
  - Summarize endpoint scoped honestly to sentence/short-passage compression (what's actually validated) — UI copy says so explicitly, not offered as full-article summarization.
  - Not yet built in this pass: notes→article and press-release→news (unchanged — still blocked on human-authored grounding data, PRD non-goal); a rigorous eval of the Research Assistant's retrieval/summarization quality at scale (only spot-checked with a few real queries so far).
- [x] **UI design-elevation pass** (visual/interaction only, no backend changes): real loading states (animated skeleton in the output panel + a spinner in the Generate button, replacing a plain text swap that left stale results visible while a new generation was in flight — a real "feels broken" bug, not just cosmetic), fade-in motion on new results (respects `prefers-reduced-motion`), a redesigned diff card for grammar output with a change count, research-card source-initial badges + hover elevation, and icon-in-circle banners for the review/flag states instead of plain emoji-prefixed text.
- [x] **Restructured `ui/rachana-desk.html` into a "Compose" unified article workspace**, alongside the original tool (kept as "Quick Tools", unchanged, via a mode toggle) — full rationale and build notes in `decisions.md`. One large auto-growing canvas for drafting/pasting an article; a whole-document toolbar limited to **headline generation only** (the one task actually trained on full-article-length input — verified before building, not assumed); a selection-tools side panel where highlighting a sentence/paragraph activates Paraphrase/Grammar/Translate/Summarize (their real validated scale) with Replace-selection/Insert-after/Copy actions; Research Assistant reused with an Insert-into-draft action; a simple undo stack. Zero backend changes needed — every endpoint already accepted arbitrary text. Verified via a Node syntax check and a full cross-reference of every DOM ID the JS queries against the HTML (all present) — **not yet visually clicked through in a real browser by either me or the user**, since no browser-automation tool is available in this environment; this is the top item to actually verify next.
- [x] **Found a real environment issue, not a code bug (2026-07-20)**: long-running background processes (the `uvicorn` dev server specifically) get killed unpredictably by this session's environment after being confirmedly healthy — reproduced 3 times, including once ~13s after a clean, verified-responding health check. Not caused by anything in `server/app.py`; server loads and serves correctly every time it's given the chance. **Workaround, not yet a root-cause fix**: run the server in the user's own terminal, outside the assistant's background-task management, for anything meant to stay up for a real session (`cd news_rachana && .venv\Scripts\python.exe -m uvicorn server.app:app --port 8000`, left running in its own window). See `errors.md`. **Not reproduced this session (2026-07-21)** — the assistant-started background `uvicorn` process stayed up and responsive for the entire session (multiple Playwright rounds, ~30+ live generations); logging this data point but not removing the workaround note above since it only takes one recurrence to matter.
- [x] **Rebuilt `ui/rachana-desk.html` into a real 3-pane Notion/Grammarly-style workspace and verified it live with Playwright** (2026-07-21) — full rationale in `decisions.md`. Persistent app shell: navy+gold branded sidebar (mode-dependent content — drafts list in Compose, task list in Quick Tools), full-height canvas, right-hand "సహాయకుడు · Writing Assistant" dock. New real feature: client-side multi-document drafts (`localStorage`, create/switch/delete/auto-save, EN+TE labeled throughout, same convention as the rest of the app). Installed Playwright into `.venv`; drove a real build→screenshot→fix loop against the live server (exp3 adapter) rather than reasoning about it statically — every screenshot this session shows real Gemma 3 generation, not mocked data.
  - **Real bugs found and fixed via the live loop** (full detail in `errors.md`): (1) the rebuild silently dropped the Undo feature (button removed from HTML, `doUndo()` deleted, but `pushUndo()` calls were still live) — restored and confirmed working via a Playwright assertion that the canvas content actually reverts; (2) a CSS cascade-order bug where the `@media (max-width: 980px)` override block, placed early in the stylesheet, was silently overridden by later unconditional rules with equal specificity — caused the sidebar to visually overlap main content at narrow widths; found precisely via `getBoundingClientRect()`/`getComputedStyle()` in a repro script, fixed by moving the whole mobile-override block to the end of the stylesheet; (3) the backend's single-GPU generation lock (`server/app.py`'s `GENERATE_LOCK`, a known/documented POC constraint) means a queued request just spins with no explanation — added a global "busy" indicator (`#busy-indicator`, sidebar footer) driven by a `pendingRequests` counter wrapped around every fetch call site, so a slow Research Assistant query queued behind another action is now legible instead of looking hung.
  - Functional pass exercised every live-model action at least once (headline/paraphrase/grammar/translate/summarize/research, Accept/Edit/Flag, selection tools, whole-doc headline, research insert, undo, draft create/delete) with zero browser console errors.
  - What did **not** change: all `/api/*` endpoints and request/response shapes, the task definitions, the guardrail banners, the diff card, and the whole-document-headline-only vs. selection-scoped-everything-else split from 2026-07-20 (still correct — this was a shell/IA rebuild, not a re-litigation of validated scope decisions).
- [x] **Newsroom-writer-POV pass: 3 features built and verified live, plus a real IndicXlit retest** (2026-07-21) — full rationale in `decisions.md`. (1) Grapheme-aware (`Intl.Segmenter`) headline character-count meter with green/amber/red zones, and a per-draft word-count target with color-coded progress. (2) A Grammarly-style floating inline selection toolbar (mirror-div caret positioning technique, no library) that shows quick action buttons near the highlighted text and an inline result+Replace card, synced with the existing dock panel rather than replacing it. (3) `server/guardrails.py` gained a general `check_fact_preservation()` (numeric + capitalized-token check) reused across grammar/paraphrase/translate/summarize, surfaced as an always-visible badge next to output (not a one-time banner) — **deliberately NOT applied to headline generation**, after the live loop showed it false-positiving on nearly every real headline (a headline is supposed to drop most numeric detail; that's correct compression, not a grounding failure). (4) Retested `ai4bharat-transliteration` on this project's actual Windows/Python 3.10 venv (prompted by a Colab notebook the user pointed at showing it works on Linux/py3.8) — new, more specific blocker found: fairseq's PyPI sdist is missing `version.txt` (real upstream defect), and installing from GitHub source instead hits `setup.py` trying to create a symlink, which needs Windows Developer Mode (confirmed OFF on this machine, a system-wide setting left for the user to flip, not toggled automatically).
  - **Real bugs found only by clicking through the live loop, not by reasoning about the code**: a `max-width` too narrow for the floating toolbar's own button row (buttons rendered outside the toolbar's background - fixed with explicit width + flex-wrap); an Escape-key dismiss silently undone by the same keypress's `keyup` re-triggering selection recompute (fixed with an explicit `dismissedRangeKey` tracker, not a bare state check); the fact-preservation false-positive on headlines described above. All three plus fixes are in `errors.md`.
- [x] **IndicXlit transliteration now actually works, via WSL2 + Python 3.10 — the Windows dead end above is superseded** (2026-07-21). WSL2 Ubuntu-24.04 (already installed, GPU passthrough confirmed with a real CUDA matmul) sidesteps the Windows Developer Mode/symlink blocker entirely. Hit and fixed two more real, previously-unseen blockers along the way: a pip≥24.1 metadata-rejection error on `omegaconf` (fixed with `pip<24.1`), and a Python-3.12-specific dataclass crash in fairseq's own config code (fixed by using Python 3.10 via deadsnakes — confirms Python *version*, not OS, was the real variable the Colab example depended on). **Verified with real generations**: `XlitEngine('te')` correctly transliterated every test word, including the two words (`రైతులు`/farmers, `ముఖ్యమంత్రి`/chief minister) that failed in the original stress test — no garbage/wrong-script output. Full writeup in `decisions.md`/`errors.md` 2026-07-21. Env: `~/venvs/rachana-translit-py310` inside WSL2 Ubuntu-24.04.
  - **Resolved 2026-07-22**: integrated directly as `server/transliterate.py` + `/api/transliterate` in the Docker deployment — no cross-OS bridge needed since HF Spaces run Linux containers natively. See the deployment entry below.
- [x] **Unsloth + Flash-Attention-2 retested in WSL2 — closed, not deferred** (2026-07-22). Same negative verdict as the native-Windows test (`decisions.md` 2026-07-19), reached via a different root cause: no prebuilt `flash-attn` wheel exists for this Python/torch/cu128 combo, and building from source needs the CUDA toolkit (not installed) plus a 30min-2hr compile with no guarantee of Blackwell/sm_120 kernel support — explicitly declined as not worth the time/risk. Real mistake caught mid-session: installing Unsloth against the project's main torch pin (2.11.0) triggered the exact silent-downgrade risk `errors.md` already warned about; fixed by installing Unsloth's own validated pin (`torch==2.10.0+cu128`) first. Training stays on the proven native-Windows stack. Full writeup in `decisions.md`/`errors.md` 2026-07-22.

- [x] **Deployed to Hugging Face Spaces — two live Spaces under `KPrashanth`** (2026-07-22). `rachana-news` (Docker SDK): full product, transliteration wired in as a real `/api/transliterate` endpoint + UI action (the WSL2-only integration question from the entry above is now resolved — Linux containers don't need a cross-OS bridge at all). Built and smoke-tested locally with real Docker before ever pushing; three real dependency bugs caught and fixed this way (see `decisions.md`). `rachana-news-demo` (Gradio SDK, ZeroGPU): validated the real technical uncertainty first via a throwaway spike (`bitsandbytes` 4-bit + `PeftModel` persists correctly across `@spaces.GPU` calls, warm-call latency ~2.5s) before building anything, then shipped 5 of 6 capabilities, all live-tested via `gradio_client`.
- [x] **Cross-checked the newsroom-writer field-report artifact against live code (2026-07-23) — found it fully accurate, then fixed 2 of its 3 recommendations.** Full rationale in `decisions.md`. (1) Grammar's Roman-script false "no changes" reassurance fixed — `factBadgeHtml()` and the diff-card label in `ui/rachana-desk.html` now check the already-returned `input_script` field and show an explicit "not processed" state instead of a green Facts-preserved badge. (2) Added a question-mark check to `server/guardrails.py:check_headline_needs_review()` — flags the exact live failure mode (headline degrades to a question, burying real news) for editor review via the existing `needs_review` UI plumbing; verified against the report's real example. (3) Research's silent ~40s wait now shows a client-side elapsed-time stage ticker (4 labels) instead of one long silent skeleton — explicitly an estimate, not real backend progress (a real fix would need restructuring `/api/research` into a stream, judged too risky to do and verify without GPU/browser access this session). **Not fixed**: headline quality itself on multi-sentence input (a model/training issue, not guardrail-fixable) — still open, see "Next up" below. This round of fixes was committed and **pushed live to the `rachana-news` Space** (commit `2f48a2d`).
- [x] **Built a reporter→editor review-queue POC (2026-07-23)** — closes the gap between the dock's long-standing "no autonomous publishing, editor approval required" notice and an actual mechanism. New `server/review.py` (append-only JSONL event log, `data/processed/review_events.jsonl`, folded by `draft_id` into current status: `in_review`/`approved`/`changes_requested`), 5 new endpoints in `server/app.py` (`/api/review/submit|queue|status/{id}|{id}/approve|{id}/send_back`, none touch the model/GPU), and a new "Editor Queue" third peer view in `ui/rachana-desk.html` (submit button + status pill in Compose, a queue list/detail/approve-or-send-back dock reusing the existing sidebar-list/main-pane/dock pattern). Guardrail flags (`needs_review`, facts-not-preserved, unsupported-script) now accumulate onto the draft and travel with it into the queue. Deliberately stops at human-in-the-loop — no auth, no CMS write, no autonomous publishing; full-product gap documented separately (see below). Planned via Plan mode first — plan file `~/.claude/plans/recursive-tinkering-twilight.md`. Committed, pushed, and **confirmed working live end-to-end via a real Playwright browser session** (submit → status pill updates → Editor Queue lists it with its flags → Approve updates status) — see the audit entry below.
- [x] **Companion full-product vision doc (2026-07-23)**, using the PRD's own Reporter/Editor/Senior-Editor/Intern persona table (`Rachana_News_PRD.md` lines 44-49) as the lens for what the POC above still needs to become a real multi-newsroom product: real RBAC, the CMS-write boundary (explicitly deferred to the PRD's already-separate Phase 3), a real database, multi-user editing/locking, notifications, audit trail/versioning, per-publication style config, ops analytics, and an explicit note distinguishing this from Rachana Data Studio (a different, training-data-curation system). Published both ways per explicit request: `docs/editorial-workflow-vision.md` (source of truth) + a Claude Artifact (https://claude.ai/code/artifact/cdd2a6a1-2ab8-4c1f-af37-25393a57d934).
- [x] **Fixed a real Research Assistant relevance bug, found by testing live queries against the live feeds (2026-07-23)** — `search_entries()` had no relevance floor, so any single matching word was enough to include an article. The UI's own default example query ("డెంగ్యూ వ్యాక్సిన్" / dengue vaccine) was live-confirmed returning a completely unrelated Ebola article, solely because "vaccine" matched. Fixed in `server/research.py` by requiring a true majority of the query's distinct words to appear before an article is even a candidate — re-verified live: the broken case now correctly returns nothing, broad topical queries (Telangana, cricket, police, government) still return solid matches. Also swapped the default example query to "తెలంగాణ" (reliably has current coverage, unlike a disease name tied to today's news cycle) and made the empty-result copy explain that this searches live feeds only, not an archive.
- [x] **Wired transliteration into Compose (2026-07-23)** — the `/api/transliterate` backend was already correct (verified live) but only reachable from Quick Tools. Compose's Grammar selection tool now shows a "Transliterate selection" button whenever the result comes back on Roman/mixed script, replacing the selection in place via the existing endpoint.
- [x] **Added real-time Roman-Telugu transliteration suggestions while typing in Compose (2026-07-23)** — a debounced check on the Latin word under the caret (not during an active selection) calls the same `/api/transliterate` endpoint for just that word and shows up to 3 candidates in a small popover near the caret (reusing the floating toolbar's mirror-div caret-positioning technique); click or Tab accepts, typing on/moving the caret/Escape dismisses, a request-sequence counter discards stale responses. Deliberately word-at-a-time on a pause, not per-keystroke live re-rendering — a true phonetic IME is a much bigger, riskier build than this POC needs. **Verified live end-to-end via Playwright**: typing "repu" → popover shows రేపు/రెపు/రేపూ → Tab accepts → typing "hyderabad" re-triggers correctly → clicking a chip also works → typing an already-Telugu word correctly shows no suggestion. Zero console errors.
- [x] **User switched `rachana-news` from private to public (2026-07-23)** — incidentally fixed a real blocker found this session: private-Space browser navigation via Playwright was hitting a 404 on every page load (an HF auth/routing quirk specific to full browser navigation, not plain API calls — a Bearer token worked fine for direct `requests` calls to the same URL the whole time). Now public, browser automation just works directly.
- [x] **Full Playwright screenshot audit against live production (2026-07-23)** — covered Compose default view, Roman-script Grammar → honest badge, the new Transliterate-selection button (confirmed it actually replaced text on production, not just that it appeared), Research Assistant results, Submit-for-review → status pill, Editor Queue → Approve. One real bug caught was in the *test script* itself (a loading placeholder shared a CSS class with the real result, so an early wait matched instantly) — fixed and reverified. Zero console errors across every screen tested.
- [x] **Cofounder-facing user manual + roadmap, requested explicitly for sharing (2026-07-23)** — `docs/user-manual.md`, covering all of the above with 8 real screenshots captured live against production (not mockups), what's deliberately not built yet, a feature roadmap, and a two-track model-improvement roadmap (RLHF/DPO on real editorial feedback once real usage exists; further task-specific fine-tuning). Also published as a Claude Artifact for easy sharing (https://claude.ai/code/artifact/991fb366-c8d8-40e7-b064-6f9f957d8394). Screenshots committed via Git LFS (`docs/manual-screenshots/*.png` — HF Spaces rejects raw binary pushes, same convention already used for the model adapter weights). Reference links given to the user: the live Space and the reference UI source file's direct HF blob URL (`ui/rachana-desk.html`, viewable now that the Space is public).

## Next up (priority order)

1. **RESUME POINT**: rerun three thin qualitative spot-checks at ~n=50 each (user-requested, paused mid-task 2026-07-22 before any generation ran — local RTX 5050/cu128 confirmed ready). Full methodology already worked out, see `decisions.md` 2026-07-22 entry so it doesn't need re-deriving:
   - Headline hallucination (was n=20) — sample fresh rows from `data/processed/exp3_eval_resample2.jsonl`, reuse `scripts/eval_adapter.py`'s load pattern, auto-flag with `server/guardrails.py:check_headline_needs_review()`.
   - Roman-Telugu robustness (was n=6) — romanize real eval sentences via local Ollama `gemma3:4b`, run through real production prompts, auto-flag with `check_fact_preservation()`/`check_headline_needs_review()`.
   - Research Assistant quality (was n=2-3 queries) — fetch live RSS entries first to design queries around real current content, since retrieval has no archive/cache.
2. Decide whether to invest in getting transliteration working on the Gradio/ZeroGPU demo Space too — needs either finding an alternative to fairseq's `pip<24.1`-dependent install path, or accepting the Docker Space as the only place it works.
3. Decide whether v0.1's settled quality profile (headline big win, translation/GEC/sentence_compression roughly flat, en→te modest dip) needs a v0.2 data-mixture fix before or after the pitch — not urgent for the POC itself, since the POC only exposes the tasks that are already solid.
4. Consider expanding the curated RSS allowlist (4 verified feeds is a thin start) — verify any new candidate feed live before adding it, same discipline as this session (2 of 6 candidates tried here didn't actually work).
5. Second automated filter pass on the GEC synthetic batch (`data/processed/gec_synthetic_pairs.jsonl`) — deferred by explicit instruction, pick up whenever prioritized.
6. Pilot the notes/press-release reverse-generation technique on ~20-30 fresh TeClass articles (script built: `scripts/pilot_notes_reverse_generation.py`, not yet run).
7. Fill in the pending manual-rating CSVs (see "Waiting on human") — closes the baseline-eval loop.
8. PRD open question #2 — seed-set staffing proposal still unconfirmed; may shrink if the notes/press-release pilot (#6) pans out.
9. PRD open question #3 — teacher-model decision, still inconclusive (TranslateGemma test).
10. PRD open question #5 — extend Rachana Data Studio for task-tagged review (needed before Phase 2, not urgent yet).
11. ~~Remaining newsroom-writer-POV gaps... reporter/editor roles + review queue + status field~~ — **built, pushed, and browser-verified 2026-07-23**, see "Done" above. Decide whether it's worth starting on any of the full-product gaps documented in `docs/editorial-workflow-vision.md` (real auth is the actual prerequisite for most of them).
12. (Nice-to-have, not requested yet) A checkpoint switcher in the UI to compare exp1/exp2/exp3 adapters side by side — the backend currently loads exp3 only at startup; would need a model-reload endpoint. Only worth building if actually needed for the pitch or for continued eval work.
13. Headline generation's actual quality drop on multi-sentence input is still open (2026-07-23 only added a review-flag heuristic, not a real fix — needs a model/training-side look, e.g. checking whether the training data's headline examples skew toward single-fact leads).
14. Research Assistant's relevance floor (2026-07-23 fix) is a real improvement but still word-bag matching, not phrase-aware — a multi-word proper noun query (e.g. "రోహిత్ శర్మ క్రికెట్") can still match an article about a *different* person with the same surname plus unrelated cricket content, since majority-of-words doesn't require the words to be about the same entity. Not fixed — would need real entity/phrase-awareness, judged over-engineering for this POC unless it turns out to matter in practice.

**Deliberately deferred**, not forgotten:
- PRD open question #4 (Phase 3 agentic/CMS layer) — already scoped separately in the PRD. The Research Assistant built 2026-07-20 is a deliberately narrow, controlled slice of this (curated RSS only, query-driven, always cited) — not a reversal of the decision to treat full agentic/tool-calling as a separate later phase.

## Waiting on human

- `data/processed/manual_ratings_paraphrase.csv` (now 400 rows), `manual_ratings_fluency.csv`, `manual_ratings_notes_to_article.csv` — need 1-5 ratings filled in
- Confirm or adjust the seed-set staffing proposal before treating it as locked
- Decide when to resume the GEC filter pass and the notes/press-release pilot (both deferred by explicit choice, not blocked on anything technical)
- Look at the rebuilt workspace yourself (`ui/rachana-desk.html`) and give real design feedback — the Playwright loop caught functional/layout bugs and iterated on a navy+gold palette, but taste calls (is this the right visual direction for a newsroom tool, any Telugu typography issues on your actual machine/fonts) are still worth a human pass
- **Set up HF billing** (see "Next up" #1) so `KPrashanth/rachana-news` can get its requested T4-small GPU tier — real account action, only the user can do this
- **Decide whether to start on any of the full-product gaps in `docs/editorial-workflow-vision.md`** (real auth first) — or the checkpoint switcher, or neither yet
- **Decide whether `rachana-news` should go back to private** now that the immediate Playwright-audit need is served — being public was the user's own call this session, not something to silently revert, but worth a deliberate decision rather than leaving it public by default
- **`data/processed/usage_feedback.jsonl` currently holds only test-harness noise, not real usage** — all 30 entries (as of 2026-07-21) are from this session's and 2026-07-20's endpoint/Playwright testing against the same canned demo article, including one literal `"input": "test in"` row from the original smoke test. No real reporter/editor has used the tool yet (per this project's scope - solo research, no newsroom partner). Decide whether to clear this file before real usage begins, so the flywheel signal isn't contaminated by test data indistinguishable from genuine Accept/Edit/Flag actions.

## Maintenance note

This file is **rewritten** each session (not appended to) — it should always reflect current state only. Session history/rationale lives in `decisions.md`; bug history lives in `errors.md`.