# AGENTS.md

> This file is the **AI-agent orchestration layer**. It is written for Codex
> Code / Claude Code / OpenCode / similar coding agents: read it top-to-bottom and execute the
> flow below to bring the full OmniDocBench v1.6 evaluation system up from zero
> on an AMD Windows machine, then verify it produces real scores.
>
> 人工介入点（⚠️）会暂停并请你确认。Exception handling points to
> [`docs/pitfalls.md`](docs/pitfalls.md) by symptom — do **not** inline fixes here.

---

## What this repo does

It builds, from scratch on **Windows + AMD Radeon GPU**, a complete
[OmniDocBench v1.6](https://github.com/opendatalab/OmniDocBench) evaluation
system: the full 1651-page benchmark, the four standard metrics (text
Edit-distance, reading-order Edit-distance, table TEDS, formula CDM), and a
reference adapter for **PaddleOCR-VL-1.6** with validated scores.

Once the infrastructure is up, evaluating *any* document-parsing model is a
matter of writing one adapter (see `adapters/_template/`). The infra is
model-agnostic; only `adapters/` is per-model.

The repo exists because the naive path hits 20+ landmines on AMD Windows
(restrictive networks, WSL Store unavailable, `\mathcolor` rendering black,
ImageMagick 6 flattening color to grayscale, two TeX Live trees disagreeing,
…). `eval-infra/02-cdm-environment/` distills the fixes into idempotent
scripts; `docs/pitfalls.md` records the debugging.

## Why an agent can run this

Every `setup.*` script is **idempotent** (self-checks before doing work) and
every setup step is followed by a **`verify.*` script that returns exit 0/1
with diagnostics**. You do not need to "know" whether a step succeeded — run
the verify, read the exit code. Re-running the whole pipeline is a no-op once
provisioned and resumes cleanly after a partial run.

---

## ⚠️ Human-intervention points

When you reach one of these, **pause and surface it to the user** with the
exact message. Do not attempt to silently work around them.

| # | When | Why human | Message to show user |
|---|---|---|---|
| ⚠️ 1 | `scripts/wsl-ensure.ps1` finishes but the distro still won't start, or it reports a missing kernel component | A fresh WSL install usually needs a **Windows reboot** before the kernel is active. An agent cannot reboot. | `⚠️ WSL installed but needs a Windows REBOOT to activate the kernel. Reboot, then re-run: wsl -d Ubuntu2204 -- echo OK. (docs/pitfalls.md#wsl)` |
| ⚠️ 2 | `01-vlm-server/setup.ps1` starts the VLM server | Confirm the **GPU is actually in use** (ROCm/HIP), and accept any first-run UAC/driver prompt. A mis-served server produces silent 0-predictions later. | `⚠️ VLM server started. Please confirm GPU utilization (e.g. rocm-smi / Task Manager) and that the server stays up, then I will continue.` |
| ⚠️ 3 | Any installer requests **UAC / admin elevation** (ImageMagick, drivers, WSL feature enablement) | An agent cannot click an elevation prompt. | `⚠️ An installer is requesting admin (UAC). Please approve it, then tell me to continue.` |
| ⚠️ 4 | `detect-mirrors.ps1` reports **no reachable source** for GitHub or HuggingFace/ModelScope | Means the machine has no working egress at all (offline, or all mirrors blocked). Needs a human to fix networking/VPN. | `⚠️ No reachable mirror for GitHub or datasets. Check networking/VPN and re-run scripts/detect-mirrors.ps1. (docs/pitfalls.md#network)` |

Everything else is autonomous.

---

## Execution flow

Work top to bottom. Run each `verify.*` immediately after its `setup.*`; only
proceed when it exits 0. All paths are relative to the repo root. Launch
PowerShell scripts with `powershell -ExecutionPolicy Bypass -File ...`.

The canonical single entry point is the profile-driven orchestrator. Three
formal profiles exist (see `scripts/profiles/*.profile.json` and
`-ListProfiles`):

```powershell
# Capability smoke on CPU (10 pages, Windows metrics + WSL CDM)
powershell -ExecutionPolicy Bypass -File scripts\reproduce.ps1 `
   -Profile cpu-smoke-10

# Fast AMD GPU acceptance (10 pages, HIP backend proof, no CPU fallback)
powershell -ExecutionPolicy Bypass -File scripts\reproduce.ps1 `
   -Profile hip-smoke-10

# Formal full benchmark (HIP, locked 1651 pages, strict gates, WSL CDM)
powershell -ExecutionPolicy Bypass -File scripts\reproduce.ps1 `
   -Profile paddleocr-vl-hip-full-1651
```

Each profile declares backend, pages, prediction dir, manifest, scoring
configs, `save_name`, port, coverage/failed-page budgets and raw-scale metric
thresholds; `reproduce.ps1` runs the same stage chain for every profile,
keyed by stable stage IDs (`environment.python`, `dataset.setup`,
`inference.server`, `inference.backend_proof`, `inference.run`,
`scoring.windows`, `scoring.wsl_cdm`, `verification.final`,
`evidence.pack`, ...). Resume re-checks the phase fingerprints — provisioning
(`fingerprint.provisioning.json`), inference (`fingerprint.inference.json`),
scoring (`fingerprint.scoring.json`) and evidence
(`fingerprint.evidence.json`) — via `scripts/compute_fingerprint.py --phase`,
and continues inference per page with `--skip-existing`. Each phase fingerprint
is computed only after its inputs are provisioned and lock-verified; a
mismatch refuses resume until a fresh run or `-ForceInference`. Formal (full)
profiles additionally fail closed on a dirty git working tree. On resume,
predictions are tree-hashed before and after inference
(`scripts/hash_prediction_tree.py`); any content change automatically
invalidates `inference.prediction_check` / `scoring.windows` /
`scoring.wsl_cdm` / `verification.final` / `evidence.pack` so stale scores are
never reused, while unchanged predictions reuse scores only after the
`<save>_metric_result.provenance.json` sidecar re-validates against the
current prediction tree (`scripts/metric_provenance.py`).
`verify_prediction_set.py` is the SINGLE source of truth for
`prediction-summary.json` (strict GT-aware summary with
`known_allowed_failures` / `unknown_failures` / `recovered_known_failures` and
the profile `allowed_failed_page_stems` allowlist); the evidence pack only
verifies and copies it.

`-Resume` reuses only passed stages whose inputs are unchanged; `-DryRun`
prints the resolved profile and stage commands; `-ListProfiles` lists the
catalog. When bulk dataset/GGUF/layout bytes already exist in another
checkout, add `-SeedFrom <path> -SkipCdmSetup`. The orchestrator verifies
source and destination locks and still creates fresh predictions and scores.

The selected profile command above owns the full flow. The numbered sections
below document its stage boundaries and diagnostic commands; do not rerun them
after a successful profile. In particular, `environment.mirrors` runs the
detector, then `environment.python` calls `Invoke-UvCatalogSync` with
`mirrors.json` and `locks\manifest.json`. A standalone root sync is not the
catalog fallback path.

### Step 0 — environment + network + WSL  (Windows)

```powershell
# 0a. Install uv before running the selected profile command above.
winget install --id astral-sh.uv -e

# 0b. environment.mirrors (run by reproduce.ps1) publishes the contracts.
powershell -ExecutionPolicy Bypass -File scripts\detect-mirrors.ps1
#   On failure → ⚠️ 4 / docs/pitfalls.md#network

# 0c. environment.python (run by reproduce.ps1) installs Python 3.11 and
# invokes Invoke-UvCatalogSync; do not replace it with a bare root uv sync.

# 0d. environment.wsl guarantees an Ubuntu 22.04 distro exists.
powershell -ExecutionPolicy Bypass -File scripts\wsl-ensure.ps1
wsl -d Ubuntu2204 -- echo OK
#   If "missing kernel component" or distro won't start → ⚠️ 1 (reboot)
#   If wsl --install hangs/fails → docs/pitfalls.md#wsl

# 0e. Select inference backend, then fail fast on selected-path prerequisites.
# Official Windows HIP releases omit Radeon 860M/gfx1152; use CPU there.
$gpuNames = @(Get-CimInstance Win32_VideoController | ForEach-Object Name)
$useCpu = ($gpuNames -match 'Radeon.*860M') -or -not ($gpuNames -match 'AMD|Radeon')
$variant = if ($useCpu) { 'cpu' } else { 'hip' }
powershell -ExecutionPolicy Bypass -File scripts\preflight.ps1 -CdmPath Wsl -Variant $variant
$repoWsl = (wsl -d Ubuntu2204 -- wslpath -a $PWD.Path).Trim()
```

### Step 1 — OmniDocBench code + dataset  (Windows, `eval-infra/01-omnidocbench/`)

```powershell
powershell -ExecutionPolicy Bypass -File eval-infra\01-omnidocbench\setup.ps1
powershell -ExecutionPolicy Bypass -File eval-infra\01-omnidocbench\verify.ps1
#   verify exit 1 + "code missing"    → re-run setup
#   verify exit 1 + "GT manifest"      → dataset download failed → pitfalls.md#network
#   verify exit 1 + "only N images"    → partial download → re-run setup (resumes)
```

### Step 2 — CDM environment (optional Windows-native path; WSL compatibility/reference path)

```powershell
# Optional native Windows CDM: run this after Step 1 only when using the
# native path. It requires local TeX Live, ImageMagick, and Ghostscript.
# WSL users can skip this verification.
powershell -ExecutionPolicy Bypass -File eval-infra\02-cdm-environment\verify-windows.ps1

# WSL CDM remains the compatibility/reference path. Step 0 computed $repoWsl.
wsl -d Ubuntu2204 bash "$repoWsl/eval-infra/02-cdm-environment/setup.sh"
# End-to-end verify: CJK formula → PDF → color PNG → CDM F1 > 0.
wsl -d Ubuntu2204 bash "$repoWsl/eval-infra/02-cdm-environment/verify.sh"
#   verify exit 1 at "IM7 not active"      → pitfalls.md#grayscale
#   verify exit 1 at "PDF→PNG"              → pitfalls.md#im7-libs or #im7-gs
#   verify exit 1 at "PNG is grayscale"     → pitfalls.md#mathcolor
#   verify exit 1 at "CJK.sty not found"    → pitfalls.md#texlive-cjk
#   verify exit 1 at "CDM F1=0"             → pitfalls.md#cdm-zero (decision tree)
```

This step is long (TeX Live 2026 + IM7 download). It is safe to re-run; each of
the 9 steps self-skips when already done.

### Step 3 — reference adapter (PaddleOCR-VL-1.6)  (Windows, `adapters/paddleocr-vl-1.6/`)

```powershell
# 3a. VLM server: selected llama.cpp build + ~1.7 GB GGUF, starts llama-server.
powershell -ExecutionPolicy Bypass -File adapters\paddleocr-vl-1.6\01-vlm-server\setup.ps1 -Variant $variant
powershell -ExecutionPolicy Bypass -File adapters\paddleocr-vl-1.6\01-vlm-server\verify.ps1 -ExpectedVariant $variant
#   hip only → ⚠️ 2 (confirm GPU in use); the -ExpectedVariant flag also runs
#   the full backend proof (assert-backend-proof.ps1): variant markers, HIP
#   binary evidence, locked tag, GPU offload and HIP/ROCm log evidence.
#   verify exit 1 / 500 errors → pitfalls.md#vlm

# 3b. Layout model: PP-DocLayoutV3 ONNX (~16 MB).
powershell -ExecutionPolicy Bypass -File adapters\paddleocr-vl-1.6\02-layout-model\setup.ps1
powershell -ExecutionPolicy Bypass -File adapters\paddleocr-vl-1.6\02-layout-model\verify.ps1
#   model not found → pitfalls.md#layout

# 3c. Install the pipeline package (clones PaddleOCR-VL-ROCm + pip install -e), then run.
powershell -ExecutionPolicy Bypass -File adapters\paddleocr-vl-1.6\00-install-deps\setup.ps1
# NOTE: --out-dir must match the prediction path the scoring configs read
# (eval-infra\01-omnidocbench\configs\v16*.yaml -> predictions/paddleocrvl_rocm).
# A different name here means score.ps1 finds no predictions and every metric is 0.
.\.venv\Scripts\python.exe adapters\paddleocr-vl-1.6\run_adapter.py `
    --img-dir  eval-infra\01-omnidocbench\data\images `
    --out-dir  predictions\paddleocrvl_rocm
#   → produces predictions\paddleocrvl_rocm\<stem>.md per page
# Verify Step 3c succeeded:
#   - predictions\paddleocrvl_rocm\ should contain ~1650 .md files (one per
#     dataset page; expect ~1651 on the full set). Check with:
#         (Get-ChildItem predictions\paddleocrvl_rocm\*.md).Count
#   - Per-page failures are caught and logged to
#     predictions\paddleocrvl_rocm\_errors.log (a few are fine; the run
#     continues). If >50% of pages fail, run_adapter.py exits 2.
#   - Expected runtime: ~seconds per page × 1651 pages (potentially hours on
#     CPU; faster on hip). The VLM server must stay up the whole time.
#   - run_adapter.py exits non-zero / few predictions → docs/pitfalls.md#vlm
#     (server down) or docs/pitfalls.md#layout (model missing).
# Validate names, UTF-8, non-empty content, error logs, and >=95% coverage:
.\.venv\Scripts\python.exe scripts\validate_predictions.py `
   --img-dir eval-infra\01-omnidocbench\data\images `
   --pred-dir predictions\paddleocrvl_rocm `
   --min-coverage 0.95
```

For the PaddleOCR official doc_parser score-comparison path, run the same
adapter with `--engine official`. That path exports evaluation-oriented
Markdown with `_to_markdown(pretty=False)` and writes to the explicit
official prediction directory:

```powershell
.\.venv\Scripts\python.exe adapters\paddleocr-vl-1.6\run_adapter.py `
    --engine official `
    --img-dir eval-infra\01-omnidocbench\data\images `
    --out-dir predictions\paddleocr_official_prettyfalse_full_2026-07-09
```

Use `-Variant cpu` instead of `-Variant hip` on non-AMD-Radeon hardware.
For the verified constrained-hardware path, run the adapter with
`--out-dir predictions\paddleocrvl_cpu_860m_200 --max-pages 200`, then follow
the exact subset build/scoring commands in `README.md`.

### Step 4 — scoring + final verification  (Windows + WSL, `eval-infra/03-scoring/`)

```powershell
# 4a. Edit_dist + TEDS pass (Windows-native, pure Python).
powershell -ExecutionPolicy Bypass -File eval-infra\03-scoring\score.ps1
# 4b. Native Windows CDM pass (requires windows-cdm.patch and a passing
# verify-windows.ps1 from Step 2).
powershell -ExecutionPolicy Bypass -File eval-infra\03-scoring\score.ps1 -Config v16-cdm.yaml
# 4c. WSL compatibility/reference CDM pass, reusing the Step 3 predictions.
wsl -d Ubuntu2204 bash "$repoWsl/eval-infra/03-scoring/score-cdm.sh" v16-cdm.yaml predictions/paddleocrvl_rocm
# 4d. Verify mandatory non-CDM metrics; zero non-CDM metrics warn but can pass.
# CDM must be positive when present or required.
powershell -ExecutionPolicy Bypass -File eval-infra\03-scoring\verify.ps1
#   Negative non-CDM metrics fail; non-positive CDM fails.

# 4e. (Optional) full chain in one command:
powershell -ExecutionPolicy Bypass -File scripts\full-verify.ps1
# Native-only verification: use -SkipVlm too when the VLM service is not running.
powershell -ExecutionPolicy Bypass -File scripts\full-verify.ps1 -SkipWsl -WindowsCdm
# Without -SkipWsl, -WindowsCdm performs dual-path verification (native + WSL).
powershell -ExecutionPolicy Bypass -File scripts\full-verify.ps1 -WindowsCdm
```

If Step 3 used the official engine, score its matching output explicitly:

```powershell
powershell -ExecutionPolicy Bypass -File eval-infra\03-scoring\score.ps1 `
   -Config v16-official-prettyfalse-full-2026-07-09.yaml
```

### Step 5 — benchmark + capability report  (Windows, `eval-infra/04-benchmark/`)

```powershell
# 5a. Single run: full pipeline with resource monitoring → capability report.
powershell -ExecutionPolicy Bypass -File eval-infra\04-benchmark\run.ps1

# 5b. Verify the benchmark produced a valid report.
powershell -ExecutionPolicy Bypass -File eval-infra\04-benchmark\verify.ps1 -ReportDir (Get-ChildItem benchmark-results -Directory | Where-Object { $_.Name -ne 'reference' } | Sort-Object LastWriteTime -Descending | Select-Object -First 1).FullName

# 5c. Stability mode: N runs for statistical confidence (recommended for reference data).
powershell -ExecutionPolicy Bypass -File eval-infra\04-benchmark\run.ps1 -Stability 5
```

---

## Exception lookup table

When a verify fails, or `verify.ps1` warns about a zero non-CDM metric, match
the symptom, read the linked
`docs/pitfalls.md` section, apply the **Fix** described there, then re-run the
verify that failed. Do not improvise a fix.

| Symptom | Read |
|---|---|
| `git clone` / `huggingface-cli` / download hangs or times out | `docs/pitfalls.md#network` |
| `uv sync --locked` says the lockfile needs to be updated under a mirror or inherited index | `docs/pitfalls.md#uv-lock-mirror-mismatch` |
| `wsl --install` hangs, or distro won't start after import | `docs/pitfalls.md#wsl` (reboot first) |
| `wsl -d Ubuntu2204` fails ("not found") but `wsl -l` shows a distro with a different name | `docs/pitfalls.md#distro-name` (name mismatch) |
| `AttributeError: ... getargspec` / `distutils` on Python 3.12 | `docs/pitfalls.md#python-version` (use 3.10/3.11) |
| CDM run exits 0 but `display_formula.CDM.all` is missing, null, non-finite, or `<= 0.0` | `docs/pitfalls.md#cdm-zero` (decision tree) |
| CDM formula PDF compiles but colors are black | `docs/pitfalls.md#mathcolor` |
| `magick --version` shows ImageMagick 6, or formula PNG is grayscale | `docs/pitfalls.md#grayscale` |
| `pdflatex: Font gkai not found` / CJK glyphs are tofu | `docs/pitfalls.md#gkaiu-map` |
| `security policy 'PDF'` from `convert` | `docs/pitfalls.md#im-policy` |
| `magick: error while loading shared libraries: libfribidi…` | `docs/pitfalls.md#im7-libs` |
| `magick` fails on PDF after IM7 install from AppImage dir | `docs/pitfalls.md#im7-gs` |
| CDM `FileNotFoundError: kpsewhich/magick/gs` on Windows | `docs/pitfalls.md#posix` (run `verify-windows.ps1`; use WSL if the native toolchain is unavailable) |
| `! LaTeX Error: File 'CJK.sty' not found` | `docs/pitfalls.md#texlive-cjk` |
| CDM passes by hand but fails under the harness (heisenbug) | `docs/pitfalls.md#two-texlive-trees` |
| `UnicodeDecodeError` mid-scoring, or mojibake in JSON/LaTeX | `docs/pitfalls.md#pythonutf8` (`PYTHONUTF8=1`) |
| `onnxruntime ... model file not found`, no predictions | `docs/pitfalls.md#layout` |
| VLM server 500 / connection refused / OOM | `docs/pitfalls.md#vlm` |
| Inference suddenly 20-100× slow, or torch crashes in `RamDb::StoreRecord` after an unclean shutdown | `docs/pitfalls.md#miopen-finddb` |
| No GPU utilization/power counters on Windows AMD (`rocm-smi` absent) | `docs/pitfalls.md#gpu-counters-windows` |
| Non-CDM metrics are exactly 0 | Usually predictions dir empty/missing/misnamed on the full dataset; check `config.prediction.data_path` matches your `--out-dir` (see Step 3c NOTE). Zero non-CDM metrics warn but can pass because tiny toy subsets can legitimately score 0. Negative non-CDM metrics still fail. |
| Benchmark report missing GPU data | `docs/pitfalls.md#benchmark-gpu` |
| `verify.ps1` exit 1 on score mismatch | `docs/pitfalls.md#benchmark-verify` |

The single most-deceptive failure is **CDM F1 = 0 with no error printed** —
*everything succeeds* (LaTeX compiles, PDF rasterizes, Python imports) yet the
score is zero. Native Windows CDM users run
`eval-infra\02-cdm-environment\verify-windows.ps1` first; WSL CDM users run
`eval-infra/02-cdm-environment/verify.sh` first. If the applicable verification
passes, CDM scoring will produce real scores.

---

## Success criteria

The system is fully operational when all criteria for the **selected CDM path** hold:

1. If using the WSL CDM compatibility/reference path, `scripts/wsl-ensure.ps1`
   → `wsl -d Ubuntu2204 -- echo OK` prints `OK`. Native Windows CDM does not
   require WSL.
2. `eval-infra/01-omnidocbench/verify.ps1` exits 0 (code + 1651 images present).
3. Applicable CDM verification passes: Windows native path
   `eval-infra\02-cdm-environment\verify-windows.ps1` prints `VERIFY OK` and
   positive identical-formula F1; WSL path
   `eval-infra/02-cdm-environment/verify.sh` prints `VERIFY OK`.
4. `adapters/paddleocr-vl-1.6/01-vlm-server/verify.ps1` exits 0 (`curl /v1/models` 200); HIP profiles additionally pass `assert-backend-proof.ps1 -ExpectedVariant hip` (binary + log + offload evidence, no CPU fallback).
5. `predictions/paddleocrvl_rocm/` contains one `.md` per dataset page (~1651);
   the `paddleocr-vl-hip-full-1651` profile's `predictions/paddleocrvl_hip_full_1651/`
   covers at least 99.8% of the 1651 pages with usable predictions (UTF-8,
   non-empty, or empty for the dataset's genuinely empty-GT pages) and at most
   `maximum_failed_pages` failures, with `_run_stats.json` selected_pages = 1651.
6. `eval-infra/03-scoring/verify.ps1` exits 0: mandatory non-CDM metrics are
   present and non-negative; zero non-CDM metrics warn but can pass; CDM must
   be positive when present or required. A selected CDM scoring path requires
   a present, finite CDM score.
7. Full reproduction runs additionally pass `scripts/verify_prediction_set.py`
   (manifest page count, coverage, failed-page budget, the
   `allowed_failed_page_stems` allowlist — unknown failed pages fail the run,
   `_run_stats.json` per-page status must match the prediction files),
   `scripts/assert-metrics.ps1` (presence, finiteness, raw scale, profile
   thresholds, freshness, full-denominator page_count, provenance sidecar
   binding, timeout/exception budgets, Windows↔WSL cross-platform agreement),
   and leave a complete evidence pack under `outputs/reproduction/<profile>/`
   (profile.resolved.json with the resolved server port, artifact-hashes.json
   incl. prediction tree + provenance hashes, canonical metrics-summary.json,
   and a strict prediction-summary.json that is never overwritten by the
   evidence pack).
8. The executable integration harness
   (`tests/test_reproduce_harness.py`, `tests/test_repro_artifacts.py`) passes:
   it runs the real `reproduce.ps1` state machine against fake scripts
   (`REPRO_TEST_HOOKS`/`REPRO_ROOT` test-only injection; the formal path is
   unchanged) and covers fresh runs, interrupt/resume, prediction-change
   invalidation, stale-provenance rejection, scoped `-ForceInference`,
   dirty-formal rejection, empty-GT and allowlist gates.
9. Releases are gated by `scripts/release-gate.ps1` (version/tag match,
   CHANGELOG, green tests, generated benchmark tables, benchmark evidence
   schema, adapter manifests, clean tree, release notes with evidence levels).
   Benchmark rows live in `benchmarks/index.json` (single source of truth;
   README tables are generated) and are labelled `smoke` / `validated resumed`
    / `clean-room` / `independent` per `docs/benchmark-evidence-policy.md` — the
    2026-08-06 full-set row (like the earlier 2026-08-03 row) is a
    **validated resumed run**, not clean-room.

Reference targets (our validated PaddleOCR-VL-1.6 results on OmniDocBench v1.6):

| Metric | Direction | PaddleOCR official engine | PaddleOCR-VL-ROCm engine | Pass threshold (reported scale) |
|---|---|---:|---:|---:|
| Text Edit-distance | ↓ | 0.03444 | 0.03402 | < 0.10 |
| Reading-order Edit-distance | ↓ | 0.12949 | 0.12824 | < 0.20 |
| Table TEDS | ↑ | 94.2393 | 94.3222 | > 85.0 |
| Formula CDM | ↑ | 96.5022 | 96.9219 | > 85.0 |

In raw `metric_result.json`, TEDS/CDM use 0-1 values, so the same threshold is
`> 0.85`.

A run whose metrics clear these thresholds reproduces our results. See
[`README.md`](README.md) for the full table vs. the official baseline.

Latest profile-driven full-set evidence (2026-08-06, Ryzen AI MAX+ 395 /
Radeon 8060S): `paddleocr-vl-hip-full-1651` passed officially with Windows
text `0.035251` / RO `0.129328` / TEDS `0.929792` and WSL CDM `0.965605`;
1649/1651 usable (2 budgeted peg-native failures, upstream
PaddlePaddle/PaddleOCR#18248; the dataset's 2 genuinely empty-GT pages accept
empty predictions per `scripts/gt_manifest.py`). Full record:
[`docs/reproduction-full1651-hip-2026-08-06.md`](docs/reproduction-full1651-hip-2026-08-06.md).
An earlier validated-resumed record (2026-08-03) is at
[`docs/reproduction-full1651-hip-2026-08-03.md`](docs/reproduction-full1651-hip-2026-08-03.md).
HIP smoke evidence:
[`docs/reproduction-hip-smoke-2026-08-02.md`](docs/reproduction-hip-smoke-2026-08-02.md).

Latest local Windows-native official-engine CDM evidence:
`C:\Users\rocm\Desktop\PaddleOCR-VL-ROCm\logs\official_cdm_rerun_20260711_092548.log`
reports Text Edit-distance `0.034`, Reading-order Edit-distance `0.129`,
Table TEDS `94.2393`, Formula CDM `96.5022`, CDM samples `2352`, metric
`timeout_case_count` `0`, exception `0`. The same log records one page-match
`quick_match_timeout` that fell back to chunked Hungarian matching.

Latest local Windows-native ROCm-engine CDM scoring evidence:
[`docs/windows-native-cdm-verification-2026-07-11.md`](docs/windows-native-cdm-verification-2026-07-11.md)
records `paddleocrvl_rocm_cdm_quick_match` on the local Windows path with
Text Edit-distance `0.0340159750`, Reading-order Edit-distance `0.1282380301`,
Table TEDS `94.3222`, Formula CDM `96.9219`, CDM samples `2352`, and
native-only `full-verify.ps1 -SkipWsl -WindowsCdm -SkipVlm` passing with
`4 passed, 0 failed, 6 skipped`.

The ROCm raw `metric_result` all-values, converted to the reported 0-100
scale, are Table TEDS `93.1345` and Formula CDM `96.7129`; the reference table
above uses OmniDocBench's official leaderboard/notebook page-level aggregation
convention.

---

## How to add a new model (after the infra is up)

You only touch `adapters/`. Five steps, documented in
[`adapters/_template/README.md`](adapters/_template/README.md):

1. `cp -r adapters/_template adapters/<your-model>`
2. Edit `run_adapter.py` — keep the signature
   `run_adapter(img_dir, out_dir, server_url)` and write
   `out_dir/<image_stem>.md` per page.
3. Edit `setup.ps1` to provision weights/server; write machine-local paths to a
   gitignored `.env.local`.
4. Run the adapter into `predictions/<your-model>/`.
5. Re-run `eval-infra/03-scoring/score.ps1` (+ `score-cdm.sh` for CDM) and
   `verify.ps1`. The scorer is unchanged — only the prediction path differs.

The reference adapter `adapters/paddleocr-vl-1.6/` is a complete, proven
example to copy from.

---

## Conventions for editing this repo

- **Idempotency**: every `setup.*` self-checks before doing work. Preserve this
  when editing — never make a setup step destructive to re-run.
- **Verify is the agent's eyes**: each `setup.*` is followed by a `verify.*`
  returning 0/1. When you add a setup step, add/extend its verify.
- **No fixes in AGENTS.md**: new landmines go in `docs/pitfalls.md` as
  `Symptom → Root Cause → Fix → Verify`, then a row in the exception table
  above. Keep this file orchestration-only.
- **Codepage**: every Windows scoring run sets `PYTHONUTF8=1`. Never remove it.
- **CDM paths**: Prefer Windows-native CDM when `verify-windows.ps1` passes after
  `windows-cdm.patch` is applied. Use WSL CDM as the compatibility/reference
  path when the native TeX Live/ImageMagick/Ghostscript toolchain is absent.
