---
name: deep-research
description: "Research the problem domain before coding. Web search for techniques, save raw sources, write structured findings, update the index."
---

# Deep Research

Research the problem thoroughly before writing code. Understand what's known, what's been tried, and what approaches exist.

## When to Use

- Starting a new task or problem
- Stuck after multiple evals without improvement
- Pivoting to a fundamentally different approach
- The problem involves domain-specific knowledge you're unfamiliar with

## Notes Directory Structure

```
notes/
├── index.md              ← table of contents for research/ and experiments/
├── raw/                  ← saved web pages, paper excerpts (immutable, never edit)
├── research/             ← your synthesized findings (link back to raw/)
│   └── _coverage.md      ← the research coverage ledger (dimensions × covered/partial/missing)
└── experiments/          ← eval reflections and results (written by reflect heartbeat)
```

## Process

### 1. Understand the Problem — and Map the Research Space

Read the task description and key files. Identify what's being optimized, what the constraints are, and what makes it hard. Check `coral log` and `{shared_dir}/notes/` for prior work.

Then **decompose the problem into 4–8 research dimensions** — the distinct things a team would need to understand to win *this* task. Derive them from the task, don't pull them from a fixed list. Useful starting prompts (not a required set): *prior art / SOTA methods*, *mechanism or theory*, *implementation / libraries*, *the evaluation & grader surface*, *failure modes*, *adjacent fields*. Drop the ones that don't apply; add task-specific ones that do.

Record them in the **coverage ledger** at `{shared_dir}/notes/research/_coverage.md` — the team's map of what's been researched and what hasn't. If it doesn't exist, create it with every dimension `missing`; if it does, read it first and target the gaps rather than re-covering what's done:

```markdown
# Research Coverage — <task name>
<!-- Owned by the research team. Update on every research pass. Dimensions are
     derived from THIS task, not a fixed list. Status: covered | partial | missing -->

| Dimension (what to understand)  | Status  | Note                                | Last touched |
|---------------------------------|---------|-------------------------------------|--------------|
| Prior art / SOTA methods        | missing | —                                   | —            |
| Failure modes of approach X     | missing | —                                   | —            |
| Evaluation surface / grader     | missing | —                                   | —            |
```

This ledger is what turns re-research into gap-targeting: every pass below updates one or more rows instead of duplicating work. `_coverage.md` is a team meta-file (the `_` prefix keeps the index/link tooling from treating it as a note) — one per task, updated in place, never forked.

### 2. Search — Cast a Wide Net, Then Focus

**Broad survey** — search for the problem class:
- `"[problem domain] state of the art methods"`
- `"[problem domain] survey paper"`
- `"[problem domain] benchmark comparison"`

**Specific techniques** — once you identify promising approaches:
- `"[technique name] vs [alternative] comparison"`
- `"[technique name] implementation details"`
- `"[technique name] python library"`

**Practical implementations** — find code and libraries:
- `"[problem] python implementation github"`
- `"[problem] open source solution"`

Do 3-5 focused searches. When reading papers and articles, focus on methodology and results tables — how did they solve it, and what performance did they achieve?

**Then take one hop out.** From your two or three best sources, follow the citation graph one step in each direction: the works they *cite* (backward — this surfaces the seminal paper the field builds on) and the works that *cite them* (forward — this surfaces the recent work that extends or contests them). Neither reliably shows up in a keyword sweep. Use `WebFetch` on a paper's reference list or its Semantic Scholar / Google Scholar "cited by" page, and fold anything new and on-topic into your set before you start writing.

### 3. Save Raw Sources — Retrieve First, Then Write

**The hard rule: a claim you can't point to a saved source for is a claim, not a finding.** Retrieve the source *before* you write the note that leans on it — even when you know the answer cold, fetching the actual page is a few seconds and it's the difference between a citation and a memory of a citation. Never write a number, a benchmark result, or a "X beats Y" into a research note that isn't backed by a file in `raw/`. The grounding check in step 6 enforces this mechanically.

For every useful source, save the raw content so it can be verified later:

```
{shared_dir}/notes/raw/source-name.md
```

Use `WebFetch` to get the full page, then write it to `notes/raw/`. These are immutable — never edit raw sources, only reference them from research notes.

When the source is **not a plain web article** (paper PDF, GitHub repo, video, conference talk, internal docs, chat log…), see [`references/source-types.md`](references/source-types.md) for capture procedure, what to extract, and the right frontmatter fields per type. Generic `WebFetch` only handles ~half of real research inputs cleanly.

When `WebFetch` fails, sources contradict, search returns nothing useful, or you find an existing-but-stale note covering your topic, see [`references/failure-modes.md`](references/failure-modes.md) for diagnosis and recovery procedures.

### 4. Compare Approaches

Identify 2-4 candidate approaches. For each, document:
- **What it is** — one-sentence description
- **Why it might work** — connection to the problem structure
- **Known limitations** — when it fails or scales poorly
- **Estimated complexity** — how hard is it to implement?
- **Evidence** — papers, benchmarks, or reasoning supporting it
- **Raw source** — link to `notes/raw/` entry

Pick your approach based on strength of evidence, implementation feasibility, and iteration potential. Proven methods beat novel ideas for first attempts.

**"No such approach exists" is a valid finding.** If the honest answer after a real search is that nobody has solved this, or the technique you expected to find was tried and abandoned, write *that* down — it saves the next agent the same dead-end search. Don't manufacture a weak candidate to fill the table; a documented negative result is more useful than a padded one.

### 5. Write Research Notes

Summarize your findings in `{shared_dir}/notes/research/`. For each technique or approach, note:
- What it is and how it works
- Expected trade-offs
- Key parameters and pitfalls
- Links back to raw sources (e.g., `see [raw/paper-name.md](../raw/paper-name.md)`)

Keep notes specific and actionable. "X might work" is weak. "X reduces Y by 30% when Z > 10 (see raw/paper-name.md)" is useful. See `references/research-note-template.md` for a structured format.

After writing or substantially updating a note, spawn the Synthesis Reviewer subagent to verify grounding. The reviewer reads the note alongside its linked raw sources and returns a per-claim verdict (`grounded` / `partially-grounded` / `inferred` / `contradicted` / `unverifiable`), saved next to the note as `<slug>.review.json` — useful because the author of a synthesis cannot objectively grade its own grounding. See [`agents/synthesis-reviewer.md`](agents/synthesis-reviewer.md) for inputs and output schema.

**Soft gate:** a note is only allowed `confidence: high` in its frontmatter *after* it has passed synthesis review — high confidence is earned by an independent grounding check, not self-declared. (The step-6 grounding check flags any `confidence: high` note that has no `.review.json`.) For `low`/`medium` notes the review is optional but still worth it when the note synthesizes 3+ raw sources, or when a later agent is auditing older notes during organize-files.

### 6. Update the Index, the Coverage Ledger, and Check Grounding

Create or update `{shared_dir}/notes/index.md`. The index only lists research notes and experiment notes — not raw sources:

```markdown
# Notes Index

## Research
- [technique-a](research/technique-a.md) — one-line summary
- [technique-b](research/technique-b.md) — one-line summary

## Experiments
- (none yet)

## Open Questions
- What hasn't been tried?
```

Raw sources are accessed by following links inside research notes, not through the index.

After writing a new note, run the link resolver so existing notes pick up cross-references to it:

```bash
python .coral/public/skills/organize-files/scripts/resolve_links.py {shared_dir}/notes/ --new <new-slug>
```

The `--new` flag scans every existing note for plain-text mentions of the new title and wraps them as `[[wikilinks]]` — without this, manual cross-referencing decays as the notes directory grows.

**Update the coverage ledger.** For each dimension you touched this pass, flip its row in `{shared_dir}/notes/research/_coverage.md` to `partial` or `covered`, link the note, and stamp the eval count. A pass that found nothing new still updates the row (it's now been *looked* at) rather than leaving it `missing`. Update in place — never fork a second ledger.

**Run the grounding check** before you consider the pass done:

```bash
python {shared_dir}/skills/deep-research/scripts/check_grounding.py {shared_dir}/notes
```

It reads only files already on disk (no network) and flags ungrounded findings: research notes that cite no `raw/` source, links to `raw/` files that don't exist, raw sources missing their `source_url` / `source_type` / `captured` frontmatter, `confidence: high` notes that skipped synthesis review, and **`_coverage.md` rows linking to a note that doesn't exist** (rename a note and the ledger silently claims coverage it no longer has). Fix what it finds — an ungrounded note pollutes the base for every agent that reads it.

It also reports an advisory `uncited-claim` count. That one is a deliberately noisy heuristic — treat it as a prompt to re-read those lines, not a gate. It is not folded into `grounding_score`.

## Maintaining Notes Across Sessions

Research notes evolve as new raw sources arrive and old ones decay. A few rules keep the synthesis honest as the corpus grows.

### Multi-source synthesis — re-write from ALL contributors

When 2+ raw sources inform the same topic, the research note must draw from **every linked source**, not just the most recent one. On a follow-up research pass that finds a new source covering an existing topic:

- **Update the existing note**, don't fork. `research/topic-v2.md` is wrong — there should be one note per topic.
- Re-read each linked raw source and rewrite the synthesis from the full set, not just the new one.
- Append the new source to the `## References` section.

If you only re-synthesize from the new source, you silently drop evidence from the old ones — which means re-research can quietly *reduce* the note's grounding instead of strengthening it.

### Stale or invalid sources — freeze, don't overwrite

When a raw source becomes invalid (link rot, retraction, supersession by a newer paper):

- **Don't rewrite the note immediately.** Add `needs-reverification: [list of claims]` to the frontmatter and move on.
- Only rewrite once you can confirm which claims survive on the remaining sources.
- If the note loses *all* its supporting sources, set `superseded: true` rather than deleting the file — it preserves the audit trail for future agents who might rediscover the topic.

### Partial re-verification preserves combined work

If a note synthesizes A + B + C and you only have time to re-verify against B, **leave the note body alone**. A partial rewrite that keeps only B's perspective drops the synthesis from A and C. Either re-verify against the full source set or note the partial check in the frontmatter (`partially-verified: [B]`) without touching the body.

## Principles

- **Retrieve first, then write** — a claim you can't point to a saved source for is a claim, not a finding
- **Save raw sources** — summaries can be wrong, raw sources are ground truth
- **Target the gaps** — read `_coverage.md` first; research what's `missing`, don't re-cover what's `covered`
- **Breadth before depth** — survey 3+ approaches before committing to one
- **Compare before committing** — always evaluate 2-4 candidates, don't latch onto the first result
- **Build on what exists** — check notes and past attempts first
- **Cite your sources** — link research notes back to `notes/raw/`, and run the grounding check before you're done
- **Don't over-research** — 3-5 searches, write notes, start coding
