# ADR-0095 — M35.1 Migration GUI design decisions

**Status:** Proposed
**Date:** 2026-05-12
**Related:** ADR-0094 (M35 migration importers), plans/migration-gui-redesign.md

## Context

`plans/migration-gui-redesign.md` lays out a 4-step wizard + redesigned
list + detail pages for M35. Review surfaced eight open decisions whose
answers shape the wizard schema, backend surface, and Phase 1 work
breakdown. This ADR locks them in so Phase 1 can start without the
plan re-litigating mid-implementation.

The eight decisions cover source-type taxonomy, polling vs streaming,
state persistence across reloads, SSRF guards on outbound discover
calls, retry semantics, and account-size discovery cost.

## Decisions

### 1. cpmove → handled inside cPanel source, not its own option

**Decision.** Step 1 shows ONE cPanel option. Step 2 carries an
ingest sub-choice (radio or tabbed): "Connect to live server" vs
"Upload cpmove backup file." Both ingest paths feed the existing
`internal/migrate/cpanel/` parser.

**Why.** Backend has a single cpanel parser; cpmove is the file
encoding cPanel uses for both backup AND live `pkgacct` output —
not a separate format. Two cards in Step 1 implies two code paths
that don't exist. Operator mental model is "I have a cPanel account"
regardless of ingest mechanics.

**Cost.** Step 2 conditional: when source=cpanel, render an ingest
toggle. ~1 day UI work.

### 2. imap_only is REMOVED from M35 entirely

**Decision.** Strip `imap_only` from the wizard AND from the
backend. Delete:

- `panel-api/internal/migrate/imaponly/` (package)
- `MigrationSourceIMAPOnly` constant (`internal/models/migration_job.go:13`)
- `imap_only` switch case in `admin_migrations.go:169 + :210`
- `imap_only` references in `manifest.go`
- UI: `imap_only` entries in `CreateMigrationDrawer.tsx:61` +
  `AdminMigrationsPage.tsx:71` + `CreateMigrationDrawer.tsx:371`

Operators who need IMAP-only mailbox sync can use the existing
M6 mailbox auto-discovery on the target side or stalwart's import
tool directly.

**Why.** Stub-grade Discoverer never made it past "not yet wired"
in the UI (per CreateMigrationDrawer:61 label). It's been
documented as a path operators can request but adds a row to every
state machine, a dropdown option in every form, and dead code paths
in every Go switch over `MigrationSourceKind`. Cheaper to remove
than to keep the surface area open for a feature that hasn't moved
in two milestones.

**Cost.** ~0.5 day cleanup + a migration to NULL out any
historically-created imap_only rows (likely zero in production).

**Reversibility.** Re-add later by restoring the package and the
constant if real demand surfaces.

### 3. Bulk-WHM is v1 must-have via job-per-account batches

**Decision.** WHM source ingests via discover-accounts (lists every
cPanel account on the source server) → operator multi-selects N
accounts → backend creates N `MigrationJob` rows, one per account,
all sharing a `batch_id`.

Schema: add `batch_id VARCHAR(26) NULL` column to `migration_jobs`
(ULID generated by panel-api on bulk create). Existing single-account
jobs leave `batch_id NULL`. New endpoint: `DELETE
/admin/migrations/batches/:id` cancels every job in a batch.

UI list view groups jobs by `batch_id` when present (collapsed by
default, expandable to see per-account progress + per-account retry).

**Why.** The WHM use case IS bulk. Single-account-at-a-time forces
the operator to repeat the wizard 50 times for a 50-account
migration. Job-per-account keeps the existing schema almost
unchanged — partial failure isolates to one row, retries are
per-account, the queue limits concurrent restores naturally.

**Alternative rejected.** One MigrationJob spanning many accounts
collapses retry/progress/error tracking into one row — fixing one
account's failed validate stage means re-running the whole batch.

**Cost.** Migration 000124-ish: add `batch_id`. Bulk-create
endpoint. List grouping in UI. Concurrency cap (env or
server_settings) so 50 jobs don't all hit `mariadb-dump` at once.
~3 days backend + ~2 days UI.

### 4. Hybrid: list page polls, detail page streams via SSE

**Decision.** List page polls `GET /admin/migrations` every 30s
(cheap, aggregate query, low frequency, no live job assumption).
Detail page subscribes to `GET /admin/migrations/:id/stream` (SSE)
emitting `{stage, state, progress, bytes_processed, total_bytes}`
events as the runner advances. Endpoint flushes a final event and
closes when job reaches terminal state.

Reuses the nginx WebSocket proxy timeouts already configured for
`/api/v1/logs/stream/*`.

**Why.** Two-mode usage. List page is "show me what's going on"
across the whole server — polling at human-noticeable cadence is
fine. Detail page is "watch this one job" — operator wants
real-time progress bar movement, not staircase jumps every 10s.

**Cost.** SSE handler in panel-api reads from migration runner's
existing per-job event channel (already exists for the runner's
own state-machine; just needs an HTTP exposer). Frontend
EventSource hook + cleanup. ~2 days backend + ~1 day frontend.

**Fallback.** If SSE proxy infrastructure breaks for an operator,
detail page silently falls back to 10s polling. Documented in
plans/m35-runbook.md (TODO).

### 5. Wizard state persists via draft MigrationJob rows

**Decision.** Add `MigrationStateDraft = "draft"` to
`MigrationState` enum. Step 1 creates the row with state=draft +
source_kind set. Step 2's credentials POST through `POST
/:id/secrets` writes to the existing encrypted-secrets path
(ADR-0094). Step 3 PATCHes `accounts` + `items` arrays. Step 4
flips state draft → pending and the runner picks it up.

Wizard URL: `/jabali-admin/migrations/wizard/:id`. Refresh restores
state from backend by fetching the draft row. Operator can close
the tab + come back tomorrow; state survives.

Draft cleanup: `jabali-migration-secrets-reap.timer` already
purges aged secrets; extend its query to also delete draft jobs
older than 24h.

**Why.** 4-step wizards on slow networks reload often. Losing 50
selected accounts at Step 3 due to F5 is the difference between
"good tool" and "operationally painful tool."

**Cost.** New state in enum + migration (additive — no data
change). PATCH endpoint on draft jobs only. Reaper SQL extension.
Wizard hook reads `:id` from route. ~2 days.

### 6. Account picker: lazy per-account size on demand

**Decision.** Discover-accounts returns username/domain/email/owner
ONLY for v1 — no `size` field. Picker shows accounts in a virtual
list; each row has an expand chevron that, when clicked, fires
`GET /admin/migrations/discover-accounts/:host/:user/size` and
displays the size inline. Operator can leave it collapsed for
fast bulk-select.

`server_settings` keyed cache (`migration_account_size_cache` table)
with TTL=24h backs the per-account-size endpoint so repeated
discovery on the same source server is instant.

**Why.** `du -sh /home/<user>` is the only reliable per-account
size on cPanel/WHM. Sequential = 50 accounts × ~5s each = 4
minutes blocking Step 3. Parallel-30 = ~30s but hammers the
source server. Optional/lazy + cached is the practical middle.

**Cost.** Discover endpoint stays simple. Lazy-size endpoint +
cache table + 24h TTL reaper. ~1.5 days.

### 7. Retry resumes from last-good stage, with explicit full-restart escape hatch

**Decision.** `POST /admin/migrations/:id/retry` resumes from the
stage AFTER the last `state=done` row in `migration_stages`. The
failed stage runs against its existing `extracted/` directory + DB
state.

UI exposes a secondary "Retry from scratch" link that nukes
`extracted/`, deletes all stage rows, and re-runs from analyze.

Backend prerequisite: every stage must be idempotent. Audit:
- `analyze` — read-only, already idempotent
- `fix_perms` — chmod/chown, idempotent
- `validate` — read-only, already idempotent
- `restore` — uses `INSERT ... ON DUPLICATE KEY UPDATE` for DB,
  `tar -xkf` (keep existing) for files, `useradd` already guarded
  by `id -u` check

Where idempotency gap is found, log + fail explicitly during
audit; do NOT silently ship resume-retry against a non-idempotent
stage.

**Why.** Re-pulling a 5GB tarball + re-extracting after a single
LDAP timeout in restore is wasteful operationally. Resume is the
right default for transient failures; full-restart is the right
recovery for data drift on source.

**Cost.** Stage idempotency audit (~1 day). Retry endpoint
(~0.5 day). UI button + Popconfirm for full-restart variant
(~0.5 day).

#### Amendment 2026-05-12 — stage idempotency audit findings

Walked each stage during the M35.1 implementation. Three stages
(`analyze`, `fix_perms`, `validate`) are read-only or use idempotent
ops (chown/chmod on the same paths). `restore` stage has ONE
non-idempotent sub-step:

**DB restore via `mysql <dbname> < dump.sql`** —
`panel-agent/internal/commands/db_restore.go:82`. The CLI mysql
client streams the dump verbatim. If the dump contains plain
`CREATE TABLE ... (` (no `IF NOT EXISTS`) and `INSERT INTO ... VALUES`,
a mid-restore failure leaves partial tables; resume-retry hits the
duplicate CREATE on the first table and bails.

Mitigations (in priority order):

1. **Operator-facing (shipped)**: UI exposes "Retry from scratch"
   as a secondary button. Documented in the ADR-0095 retry-resume
   description that DB-restore failures should use from-scratch.
2. **Backend (M35.2)**: db.restore handler issues `DROP DATABASE IF
   EXISTS <dbname>; CREATE DATABASE <dbname>;` before streaming the
   dump. Destructive on the destination DB but the destination is
   always a freshly-provisioned target (per M35 spec — operators
   restore INTO new accounts, not over existing ones).
3. **Source-side (out of scope)**: have mariadb-dump emit
   `--add-drop-table` on the source. Operators don't control that.

Other restore ops are idempotent:
- `rsync -aH --delete-after` syncs the dest tree to match source
- `chown -R` is repeat-safe
- `useradd` already guards via `id -u` check
- Mailbox JMAP `Manual` upserts existing folders

Retry-resume is therefore SAFE for failures in analyze/fix_perms/
validate stages. For restore-stage failures, the UI's "Retry from
scratch" path remains the operator-recommended action until M35.2
adds the DROP+CREATE bracketing.

### 8. SSRF guards: deny private + DNS rebinding protection

**Decision.** Every outbound dial from `internal/migrate/`
(discover-accounts, secrets push, pull-source) goes through a
new `internal/migrate.ResolveAndValidateHost(host)` helper that:

1. Resolves the hostname to ALL A/AAAA records.
2. Rejects if any resolved IP falls in:
   - 127.0.0.0/8 (loopback)
   - 169.254.0.0/16 (link-local, cloud metadata)
   - 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 (RFC1918)
   - ::1/128, fc00::/7, fe80::/10 (IPv6 equivalents)
3. Records resolved IPs in a per-dial context; the dial fn uses
   `(*net.Dialer).Control` to assert the connected peer's IP
   matches the resolved set — defeats DNS rebinding (record
   flips between resolve and connect).

Override flag: `server_settings.migration_allow_private_hosts
BOOLEAN DEFAULT false`. When true, the RFC1918 rejection is
bypassed (rebinding check still applies). Documented in
runbook as "ON only for VPN-tunneled internal migrations from
legacy infra; OFF in any setting where panel-api's network can
reach untrusted internal services."

**Why.** Discover-accounts without guards = SSRF primitive
operator-exposed in the admin UI. Cloud metadata service access
via this path = critical sev. RFC1918 may be legit for
internal-network migrations but the default must be safe.

**Cost.** ~1.5 days: helper, dialer integration in every
outbound caller, setting toggle, runbook entry, tests covering
each blocked range + rebinding.

## Consequences

### Backend changes summary

- Add `batch_id` column to `migration_jobs` (migration 000124)
- Add `migration_account_size_cache` table (migration 000125)
- Add `MigrationStateDraft` constant + transition rules
- New endpoints:
  - `POST /admin/migrations/bulk` (returns batch_id + N job_ids)
  - `DELETE /admin/migrations/batches/:id`
  - `GET /admin/migrations/:id/stream` (SSE)
  - `PATCH /admin/migrations/:id` (draft state only)
  - `GET /admin/migrations/discover-accounts/:host/:user/size`
  - `POST /admin/migrations/:id/retry`
  - `POST /admin/migrations/:id/retry?from_scratch=true`
- Remove `imap_only` source kind + all references
- Stage idempotency audit + fixes
- SSRF helper + dialer wiring + `migration_allow_private_hosts` setting
- Extend migration-secrets reaper to delete draft jobs >24h old

### Frontend changes summary

- Wizard URL-based state restoration from draft job
- SSE EventSource hook for detail page
- Source list collapses to 4 (cPanel / DirectAdmin / HestiaCP / WHM);
  Step 2 has cpanel ingest sub-toggle
- Account picker with lazy size column
- Batch grouping in list view + per-batch cancel button
- Retry button with full-restart secondary action

### Schedule impact

Plan's 4-week estimate assumed zero backend surface changes. Actual
work breakdown after these decisions:

- Backend SSRF + draft state + batch + SSE + size cache + retry: ~10
  days
- Frontend wizard + list + detail rewrite: ~10 days
- Stage idempotency audit (separate, can parallel): ~3 days
- E2E + runbook: ~3 days

Realistic Phase 1+2 timeline: **6 weeks**, not 4. Phase 3+4 stay at
~2 weeks combined for polish/responsive.

### Risks

- Stage idempotency audit may reveal a non-trivial gap (e.g. nginx
  vhost generation overwrites operator edits). Fixing that is out of
  M35 scope but blocks decision 7 — fallback is shipping with
  full-restart-only retry until the gap is closed.
- SSE through nginx proxy needs `proxy_buffering off` per location;
  re-check the existing logs-stream config covers /migrations/:id/stream
  by glob or requires its own block.
- Removing `imap_only` (decision 2) is irreversible at the schema
  level only if there are existing rows; verify in prod via
  `SELECT COUNT(*) FROM migration_jobs WHERE source_kind='imap_only'`
  before shipping the cleanup migration.

## Alternatives rejected

| Decision | Rejected option | Why |
|---|---|---|
| 1 | cpmove as own source | Duplicates cpanel parser; confuses operator |
| 2 | Hide imap_only behind feature flag | Keeps dead code paths; better to delete |
| 3 | v2 punt on bulk WHM | WHM use case IS bulk; v1 without it fails the primary user |
| 4 | Pure SSE everywhere | List page doesn't need real-time; polling cheaper |
| 4 | Pure 10s polling | Burns load on viewer count + jumpy progress UX |
| 5 | sessionStorage only | Lost on tab close; doesn't survive crash |
| 6 | Sync size fetch | 30s–4min blocking on Step 3 unacceptable |
| 7 | Always-full-restart | Wasteful re-download on transient failures |
| 8 | Operator-managed allowlist | Operationally heavy; operators will disable |

## Implementation order

Phase 1 (Week 1-2): decisions 1, 2, 5, 8 — they shape the data model
and wizard shape; defer until done.

Phase 2 (Week 3-4): decision 3 (batch) + decision 6 (lazy size).

Phase 3 (Week 5): decision 4 (SSE) + decision 7 (retry).

Phase 4 (Week 6): polish, mobile, E2E.
