# Test Plan: SDD Dispatch-Content Recipe

## Step 0: Classify the failure (before any test)

The proposed addition is a **positive recipe** — "your dispatch should contain (1)…(5), in order." Per *Match the Form to the Failure*, recipes are the right form for a **wrong-shaped-output** failure: the agent complies (it writes a dispatch) but the dispatch has the wrong shape (bloated, missing the brief pointer, restates the spec, buries the report path).

So the verification must confirm two things:
1. **A real failure exists** without the guidance (otherwise don't deploy — Iron Law: no skill change without a failing test first).
2. **The recipe wording actually produces the five-part shape** under a competing incentive, beating the no-guidance control.

This is a wording/shaping problem, so I lead with **micro-tests against a no-guidance control**, not full pressure scenarios. I do NOT skip the control.

## Step 1: Establish the realistic context

The guidance lives inside the SDD skill that an orchestrating agent has already loaded. So:

- **System prompt** = the full SDD skill *as it stands today* (the realistic context), plus, for treatment variants only, the proposed addition spliced in where it would actually live.
- **User message** = a concrete task that *tempts the failure*: dispatch a subagent for one task in a multi-task plan, with a competing incentive to make the prompt self-contained.

The tempting incentive is essential — without pressure, agents produce clean output and the guidance looks fine even when it isn't binding.

## Step 2: Define variants

| Variant | System prompt contains | Purpose |
|---|---|---|
| **A — Control** | SDD skill *without* the addition | RED baseline: does the failure even occur? |
| **B — Proposed recipe** | SDD + teammate's draft verbatim | Does this exact wording fix it? |
| **C — Recipe, brief-pointer as REQUIRED slot** | SDD + draft rephrased so (2) is a structural required field | Compare forms if B underperforms on the brief pointer |

C is a backup variant I run only if B fails on element (2); I don't author it speculatively, but I plan for it because "the brief is your requirements" is exactly the kind of element that gets omitted and is better enforced structurally than by prose.

## Step 3: Scenarios (user messages)

I want variety so I'm measuring the wording, not one lucky prompt. 3 distinct task scenarios, each engineered to tempt a *different* failure mode:

1. **Self-containment temptation** — "the subagent won't have repo access, so make the prompt complete." Tempts restating the brief inline instead of pointing to it (attacks element 2).
2. **Sunk-context temptation** — orchestrator just finished a long design discussion. Tempts dumping that narrative instead of the five parts (attacks brevity + report path getting buried).
3. **Ambiguity present** — the task has a genuine spec ambiguity. Tempts either silently resolving or punting (attacks element 4).

## Step 4: Sample sizes

- **5 reps per variant per scenario.** Single samples lie; 5 lets me read variance.
- Control + B across 3 scenarios = 30 calls. C only if triggered.
- One API call per sample, fresh context each.

## Step 5: Scoring

Per-dispatch, I score each of the five elements present/absent, **reading every dispatch manually** — automated keyword matching produces false hits (a dispatch that *quotes* "read this first" while still restating the whole brief looks compliant to a regex). Specifically:

| Element | Pass condition (judged by reading) |
|---|---|
| 1. Placement | One line situating the task; not a paragraph |
| 2. Brief pointer | Brief path given AND framed as the requirements AND brief NOT restated inline |
| 3. Interfaces | Cross-task interface info present (or explicit "none needed") |
| 4. Ambiguity resolution | Ambiguity surfaced + resolved (scenario 3 especially) |
| 5. Report path | Report-file path present and findable, not buried |

Plus two shape metrics:
- **Bloat**: word count of dispatch; flag restatement of brief content.
- **Order/variance**: do the 5 reps converge on the same structure, or scatter into 5 interpretations?

## Step 6: Acceptance criteria

**Gate 0 — failure exists:** Control (A) must exhibit the failure (missing brief pointer, bloat, or buried report path) in a clear majority of reps. **If the control is clean, STOP — do not deploy the addition.** There's nothing to fix.

**Gate 1 — recipe binds:** Variant B must, across all 3 scenarios:
- Produce all five elements in ≥ 4/5 reps per scenario.
- Element 2 (brief-as-requirements, no inline restatement) pass in ≥ 4/5 — this is the element most prone to negotiation under the self-containment incentive, so I hold it to the same bar explicitly.
- **Low variance:** reps converge on the same shape. Five different interpretations = wording not binding → tighten before deploying, don't just add words.

**Gate 2 — beats control:** B's element-presence and bloat scores must beat A head-to-head. A recipe that doesn't outscore no-guidance isn't earning its tokens.

**If B fails only on element 2:** run variant C (structural REQUIRED slot) and re-apply Gates 1–2. This follows *Match the Form to the Failure*: an omitted-element problem is better fixed by a required structural field than by a prose list item.

## Step 7: After micro-tests

Micro-tests verify wording, not discipline. Since this is a shaping skill (not a rule an agent skips under pressure), micro-tests + the manual reads ARE the primary gate here. I do **one** full end-to-end SDD dispatch with the winning variant to confirm it composes with the rest of the skill in a real run before committing.

## What I explicitly will NOT do

- Won't deploy because the recipe "obviously reads well" — clear to the teammate ≠ binding on other agents.
- Won't trust automated match counts — every flagged dispatch read by hand.
- Won't skip the control — without it I can't tell if the recipe fixes anything or the failure was never there.