# SDD ledger — plan: docs/plans/2026-07-28-codex-efficiency-evals.md
Task 1: complete (commits 42892fa..19dfe8e, review clean)
Task 2: ruling — reviewer's event_id KeyError finding governs over brief's verbatim sample (plan Global Constraint: parser trusted on real corpus); fix dispatched
Task 2: fix round 1/5 (2 addressed, 0 open; commits db3df44..e124a8c)
Task 2: complete (commits 19dfe8e..e124a8c, review clean after fix round 1)
Task 3: fix round 1/5 (1 addressed, 0 open — spawn_calls broad classifier; commits 791c35d..cce1aec)
Task 3: complete (commits e124a8c..cce1aec, review clean after fix round 1)
Task 4: complete (commits cce1aec..ab48efd, review clean) — parser TRUSTED; restriction: no absolute-count corpus comparisons for window-spanning sessions
Task 5: fix round 1/5 (1 addressed — gauntlet blinding; commits ec7c603..292b548)
Task 5: complete (commits ab48efd..292b548, review clean after fix round 1). Smoke rep cost $4.45. Ledger note: uncommitted fixture-gen script replicates codex provision logic (drift risk, scratch-only)
Task 6: ADJUDICATION (Jesse-approved) — E1 split: axis A model-explicitness discriminates (0/34 explicit on dev), treatment battery proceeding; axis B fork-isolation inconclusive-by-zero in fresh sessions, RE-SCOPED into E6 (long-history/compaction elicitation). Controller independently verified 34/34 fork_turns=none, 0/34 model via raw grep.
Task 6: complete (commits 292b548..927b87c, review clean). E1 verdict: baseline 34/34 isolated forks (gate fail on fork axis), axis A inconclusive-by-infrastructure (container CLI 0.144.4 < 0.145 needed for model params). Spend to date $41.87.
AMENDMENT queued: Task 6b (container CLI upgrade to field version + E1 axis-A rerun + baseline version-sensitivity check), Tasks E7-E9 (Drew-derived MINE scorers: wait-polling, close_agent, workspace leaks), Drew-corpus cross-validation. Drew dataset: /Users/jesse/git/superpowers/_tmp/drew-sdd-head-to-head-2026-07-27 (never commit).
Task 6b: fix round 1/5 (1 addressed — score_e1 rep-range names + overwrite guard; commits 3890af5..f5f1b30)
Task 6b: complete (commits 15992dc..f5f1b30, review clean after fix round 1)
E1 FINAL ADJUDICATION: at field CLI 0.146, fresh-session dev shows NO pathology on either axis (14/14 isolated, 14/14 explicit model); axis A does not discriminate arms (spinout 31/33 explicit). E1 terminal per discrimination rule. Both axes + the depth-2 lead (2/2 depth-2 spawns model-omitted, 1 fork=all) transfer to E6's long-history condition. E1 spend $66.88.
Task 12 (Drew cross-validation): complete (commits f5f1b30..f323192, review clean). 121 spawns reconciled; parser findings: multi_agent_v1 schema lacks fork_turns/sub_agent_activity (affects codex-5_5-era rollouts only). Unsourced '0/53' replaced with computed 0/64 reviewer no-recursion.
Task E7: complete (commits f323192..a80db18, review clean). Verdict: wait-timeout waste is STRUCTURAL — 78.3% stress / 74.5% audit root / 60-70% even in our small fresh batteries. Minor (deferred): score_e7 lacks FORCE overwrite guard; rebill estimate beyond literal bullet (pre-registered).
Task E8: complete (commits a80db18..c2d6516, review clean). Verdict: closure behavior is BINARY — codex-5_5/multi_agent_v1 closes 18/18, every other controller in all corpora closes 0 (incl. our fresh batteries 0/48, 0/67). Open question: why the v1 exception (schema? model?).
Task E9: fix round 1/5 (1 addressed — review-package surface; commits 9b59264..d45ebcc)
Task E9: complete (commits c2d6516..d45ebcc, review clean after fix round 1). Verdict: Drew fractals 2/4 leaked (his claim corrected from 3/4 via his own scope-audit), all leaked-then-removed; our batteries 0/14 (prediction miss — plausibly the standing workspace .gitignore, unproven). Review-package surface: scored zero both corpora.
Task 7: complete (commits d45ebcc..78ee40e, review clean). E2 micro: inconclusive-by-zero (0/20 spawns all variants incl. null; 20/20 bug found) — FULL scenario carries the baseline question.
Task 8: complete (commits 7e6d75a + 3e470d0, review clean). E2 FULL: inconclusive-by-zero — 4/4 reps root spawns exactly one reviewer child, zero nonroot spawns; folded into E6 per pre-registration. Seeded docstring bug 4/4 caught; seeded coverage-gap 0/4 (weak probe, note for E5 design). E6 now carries E1 axes + E2 baseline question.
Task 13 (E1-v611): complete (commits 981b89a + c1e8d1f, review clean). Verdict: v6.1.1 at CLI 0.146 = clean (22/22 isolated+explicit root spawns) — skill-version dependence ELIMINATED; long-history theory is the only live explanation. Depth-2 hygiene-drop pattern now 3 occurrences across skill versions (E6 lead). Lane B infrastructure proven (auto-named containers; bun bind-mount workaround: rsync node_modules).
Task 15: complete pending review (commit ed8e9a1) — 07-29 corpus ABSENT from this machine (3-way search + archived_sessions negative); all 7 claims UNVERIFIABLE, adapter ready if corpus surfaces. CONTROLLER CORRECTION: my earlier 'both rollouts exist' check was wrong — awk-mangled ls error read as success. Claims downgraded to narrative-tier evidence until corpus located.
Task 15: fix round 2/5 (1 addressed — _pick_root all-legs + env override, 11 tests; commits 4f45669..4f92e62)
Task 15: complete (commits 6c0f827..4f92e62, review clean after 2 fix rounds). VERDICT: all 7 audit claims UNVERIFIABLE — corpus absent from every reachable host (remote-host-b, remote-host-a, remote-host-c, remote-host-d; remote-host-g blocked on host key; remote-host-e/remote-host-f unreachable). Adapter + env override ready for instant reconciliation when corpus surfaces. Claims remain narrative-tier.
Task 15: fix round 3/5 (1 addressed — docstring provenance; commit c4c2a5d)
Task 15 FINAL: complete (commits 6c0f827..c4c2a5d, review clean after 3 fix rounds + reconciliation). RECONCILED on remote-host-a corpus: 6/7 claims exact (193 waits, 24 list_agents, 148 test invocations w/ exact per-agent split, depth-2 implementer-spawned reviewer + controller duplicate, 9v4, 14 sessions); 12x cluster is actually 9x; NEW FINDING: audit fabricated its 2 evidence citations while substance held. Privacy sweep adversarial-clean.
Task 11 (E4): minor (deferred): score_e4.py written before its tests — TDD-ordering deviation, compensated w/ verified mutation checks (reviewer-confirmed adequate)
Task 11 (E4): complete (commits 2e0f543 + 79e5631 + 916c450, review clean). Verdicts: spike-vs-arch inconclusive-by-zero (spike never patches — structural); bounded-vs-arch partial scaling w/ invariant two-doc ritual; MICRO: current hard-gate text forces bounded→FULL 5/5 while Z-null and B-three-path both differentiate 5/5 — the LITERAL TEXT is the lever. Outage cost $4.54 (2 tainted arch reps). Cumulative spend $104.45. Scenario-authoring lesson: post-checks must not assert measured behavior (2nd occurrence).
Task 9 (E6): complete (commits 22a4c72, 3d3faf0, 89aacfe, 16410ed, 99c5ad7, 51607a9). Mechanism: model_auto_compact_token_limit config knob (source-verified against codex-rs, 2 calibration runs, both 100% reliable), threshold 40000 derived from Task 6's real token curve. Baseline (dev, 3 reps) + treatment (spinout, 3 reps) batteries: 6/6 show >=1 real compaction. Split verdict (E1-axis-split character): clause (a) re-reads fails on dev (0/3) but corroborated on spinout (1/3, hook firing live) and Drew's stress-2703 (1 organic re-read, no hook); clause (b) degradation holds narrowly on dev (1/21 post-compaction spawns, breaking a prior 14/14 dev-arm explicit-model streak — 45/45 is the both-arms figure; corrected in the fix wave), absent on spinout (0/21). Depth-2-implementer-spawned-reviewer + same-task-duplicate-review pattern now confirmed 9x across 4 corpora/sources (corrected in the fix wave from 11x/5 — the E1 re-test and the cx-sdd-small re-score are the same runs), 0 counter-examples — most-reproduced finding in the campaign. Real bug caught+fixed via manual verification (task_family() missed "task1_controller_review" wording, commit 99c5ad7). Privacy: 2 Drew task_name-string citations caught and fixed before commit (never-commit rule). Spend ~$26.12; cumulative ~$130.57.
Task 9 (E6): review found CRITICAL — a real Drew stress-corpus task_name (the string itself is deliberately not repeated in this now-tracked file) committed in test_score_e6.py (99c5ad7) + false no-leak/scoped-out claims; fix round 1 dispatched. NOTE FOR BRANCH-FINISHING: leaked string persists in git history at 99c5ad7 — Jesse decides history rewrite vs accept at merge.
Task 9 (E6): minor (deferred→fixing): pre-reg commit 6 min after dev-rep1 session start (before any verdict)
Task 9 (E6): fix round 2 dispatched — correction prose itself re-leaked a 2nd Drew task_name (aeb77e6 commit msg + log + report); BRANCH-FINISHING FLAG updated: TWO history-embedded strings (99c5ad7 tree, aeb77e6 commit message) pending Jesse's rewrite-vs-accept decision.
Task 9 (E6): fix round 3/5 (1 addressed — scratch-report token de-leaked, substring-grep 0/84; no commit needed)
Task 9 (E6): complete (commits 22a4c72..98aef5d, review clean after 3 fix rounds). VERDICTS: (a) re-reads absent in short dev scenarios, hook fires on spinout, 1 organic in stress corpus; (b) post-compaction spawn hygiene holds (1/21 model-omission dev, 0/21 spinout, isolation never degrades); (c) depth-2 implementer-spawned pattern 9/9 across 4 corpora, all 9 also same-task duplicate reviews (corrected in the fix wave from 11/5 and 9/11) — STRONGEST CAMPAIGN FINDING; (d) forced compaction adds overhead, completion 6/6. Privacy saga: 3 leak instances across fix rounds, all editable surfaces now verified clean w/ substring-aware sweeps; 2 history-embedded strings flagged for Jesse at branch-finishing.
Task 14 (E10): minors (deferred): per-line negation heuristic in score_e10 file-claims; two commit-message count nits; '0 false claims' phrasing vs 1-unverifiable in scorer output
Task 14 (E10): complete (commits 85725ce..96d8afc, review clean). VERDICTS: all 3 engineered probes missed predictions in the HONEST direction (killed child -> honest FAIL 2/2; empty-output -> blocker surfaced 2/2; budget-cut -> judge says investigate); citation integrity 100% on 38-run fresh corpus (contrast: 07-29 long session fabricated citations). Scoping finding: Finding 7's false-success pathology requires Codex-internal soft budget exhaustion, unreachable by external probes. Spend $9.20, cumulative $139.77.
Task 10 (E3): fix round 1/5 (2 addressed + minors; commits b21fbf6 + 193167c). Re-review PASS w/ pre/post causal reproduction.
Task 10 (E3): parked — 193167c bundles E5's RESULT log entry preceding its backing files (031937f) — ruling: historical ordering anomaly from shared-working-tree concurrency, backing files present at HEAD, disclosed in both task reports; no rewrite. Root cause = my concurrency model; noted for closeout.
Task 10 (E3): complete (commits c78149e..193167c, review clean after fix round 1). VERDICTS: fresh finishing flows show NO duplicate gates (0/3 — reruns merge-justified) and RESPECT waivers (0/2, scoped re-verification); duplicate-gate pathology confirmed 1/23 in SDD loops + 07-29 corpus (9x cluster validated); invalidation guard PASSES on dev (regression baseline for receipts treatment). De-escape fix corrected MINE 5/23→1/23; parity contract preserved. Spend $3.69+$~1 probe.
Task 12 (E5): fix round 1/5 (2 addressed; commit 3d473bd, re-review PASS)
Task 12 (E5): complete (commits 2a6afe1..3d473bd, review clean after fix round 1). VERDICTS: crisp seeds caught 3/3 (scope-blindness inconclusive-by-zero by design tradeoff); serial remediation 2/3 reps (0,1,2 cycles); duplicate reviews 0/3; criterion-less gate findings 0/13; accretion = scripted fix only. D4 scope classifier has compound-exec limitation (flagged).
ALL EXPERIMENTS COMPLETE. Entering finish sequence.
FINAL WHOLE-BRANCH REVIEW: MERGE AFTER ONE FIX WAVE — no finding invalidates a verdict, scorer, or battery. 2 critical (E6 depth-2 tally 11→9 + dedupe; 45/45 = combined, dev alone 14/14), 3 important (budget ledger incomplete/chain wrong; disclosures living only in this gitignored workspace; Task 13 closeout undelivered), 8 minor (M1 score_e7 FORCE guard = FIX; M3/M7/M8 + E4 TDD-ordering + E10 negation-heuristic/count nits = ACCEPT; M4/M5 DESIGN.md staleness + M6 terminology collisions = FIX).
FIX WAVE (2026-07-30): all of the above addressed. C1/C2 corrected in out/e6-report.md + appended log correction (this ledger's lines 32/37 corrected too). Budget ledger completed (E6 batteries, E10 probes, E4 micro marked unmeasured, E2 micro marked subscription-billed, MINE work $0) — true total ≈$153.2, of which $152.34 measured. 193167c ordering anomaly + this ledger's tracked status + the two history-embedded strings disclosed in an appended tracked log entry. NEW FINDING while sweeping this file for the force-add: line 33 still carried the literal stress-corpus task_name — redacted before tracking (the gitignored workspace had been treated as out of scope for the privacy rule, which was wrong the moment we decided to track it). String-scope re-audit found the two history strings are NOT equally sensitive: string 1 is stress-run-distinctive across six commit trees; string 2 is a generic SDD label also present in our own corpus, leaked only as an attribution in aeb77e6's message.
LEDGER NOW TRACKED: force-added (git add -f) as the campaign's decision record, controller-approved exception to the workspace-cleanup convention. Anything appended here from now on is public.
CAMPAIGN COMPLETE 2026-07-30: final whole-branch review + one fix wave + re-review = MERGE-READY at 06f024d. Closeout: reports/2026-07-codex-efficiency-campaign.md. Awaiting Jesse: merge decision, history-string decision, promotion push approval.
