# Merged records — investigation artifacts, NOT shippable

⚠️ **These records validate cleanly and must not be used as datasheets.** They
carry no provenance, and one of them is knowingly incoherent. They exist as
evidence for the coverage-merge investigation in #176.

Produced 2026-07-29 from the three generic baseline replicates
(`2026-07-28_claude-opus-5-generic_rep{1,2,3}`) by
`src/data_sheets_schema/merge.py`.

## Why they cannot ship

`provenance.py` has no way to say "this record consumed other generated
records", so a merged record cannot honestly claim `record_mode: live` — and
claiming it would be exactly the unobserved provenance assertion that module
exists to prevent. A `derived_from` block naming every contributing label and
md5 is a prerequisite. See NEXT_TASKS.md item 3.

The playbook also forbids cross-label reads in four places, and a merged record
is cross-label by definition. That needs an explicit carve-out rather than a
quiet exception.

## `2026-07-29_coverage-union/` — unguarded, CM4AI is incoherent

Kept because it demonstrates the failure mode. The CM4AI record says:

    title    Cell Maps for Artificial Intelligence (CM4AI) Data Release Programme
    version  June 2026 Data Release (Beta); Dataverse dataset version 2.0
    issued   2026-06-17T00:00:00Z
    doi      doi:10.18130/V3/HIGT4C

The title came from rep3, which describes the release *programme*. The version,
issue date and DOI came from rep2, which describes one release inside it. The
record documents neither subject.

**It passes `linkml-validate`.** That is the point of keeping it: schema
validation cannot detect a spliced referent, so a separate coherence check is
required and no amount of stricter validation substitutes.

`union_merge(guarded=False)` now raises on records like these rather than
writing them, so this directory cannot be regenerated by the current code.

## `2026-07-29_guarded-union/` — referent-safe

Every referent-bearing field comes from one base record; identities the base
never asserted are dropped rather than imported.

| project | base | coherent | merged | best single | gain | fitness sum | mean |
|---|---|---|---|---|---|---|---|
| AI_READI | rep3 | yes | 81 | 80 | +1 | +4.26 | +0.0419 |
| CHORUS | rep1 | yes | 59 | 57 | +2 | +3.65 | +0.0332 |
| CM4AI | rep3 | no — guard active | 70 | 69 | +1 | +3.59 | +0.0392 |
| VOICE | rep2 | no — guard active | 83 | 79 | +4 | +5.56 | +0.0232 |

All four validate against `Dataset`.

The guard costs only what it should: on the two coherent projects it reaches the
full union (81/81, 59/59), and it gives up ground only on CM4AI — 70 rather than
73 — dropping precisely the three asymmetric fields that made the unguarded
record incoherent.

## The result worth carrying forward

The coverage gain is small: +1 to +4 slots. **The mean fitness gain is not**, and
it is uniform across all four projects (+0.023 to +0.042).

So most of the value is not the extra slots. It is choosing the best-fitting
variant among the ~57 slots the replicates *share* — replicates differ in
coverage, but they also render shared fields at measurably different quality, and
a merge captures that where selecting one replicate cannot. That is a different
justification for merging than the one #176 originally proposed, and unlike the
coverage argument it survives the referent guard intact.
