# Building a DEMON frontend (agent instructions)

You are building a UI against a DEMON realtime-music pod. Everything you
need is in this directory (`demon-client`) plus two HTTP manifests the
backend serves. Do not reverse-engineer the demo app in `demos/realtime_motion_graph_web/web/` — build
from the contract.

## Step 0: discover, don't hardcode

The control surface is backend-owned and self-describing. Fetch both
manifests at boot and build your UI from them:

| Fetch | Returns | Use it for |
|---|---|---|
| `fetchWireContract()` → `GET /api/protocol` | `{version, commands, events, config, handshake}` | Every command you may send, every event you'll receive, the session-init config schema. |
| `fetchKnobManifest()` → `GET /api/knobs` (`?sde=1` for SDE mode) | `{version, knobs}` | The `params` payload: render each knob by `type` (float/int → slider, enum → select, bool → checkbox), group by `group`, bounds/options/defaults from the entry. |

Compile-time: import the generated types from `types/wireContract.gen.ts`
(via the package root). Compare `PROTOCOL_VERSION` and
`KNOB_SCHEMA_VERSION` against the served manifests' `version` fields and
warn on mismatch — that means your build predates a contract change.

Never declare a knob list, a command shape, or an enum option list by
hand. If you find yourself typing `"denoise"` with a hardcoded max, stop
and read it from the manifest instead.

## Step 1: the minimal working session

Follow the Quickstart in [README.md](./README.md) — it is accurate and
complete: build a `SessionConfig` (schema = the contract's `config`
section), `new RemoteBackend(wsUrl, pcm, channels, config)`, `await
remote.connect()`, `new AudioPlayer()`, `init` + `resume`, then drive it
with `sendParams` and the typed `send*` methods. The pod serves HTTP and
WS on one port (default 1318).

**Playing a fixture the pod already has?** Set `use_server_fixture: true`
(+ `fixture_name`) in the config — the server loads the waveform off its
own disk and the binary PCM upload is skipped entirely. Don't download a
fixture in the browser just to re-upload it; that wastes megabytes and
seconds on every session start. Same idea mid-stream:
`sendSwapSourceByName` / `sendSetTimbreFixture` / `sendSetStructureFixture`
instead of their PCM-upload variants whenever the audio is already
server-side.

## Invariants the manifests can't tell you

1. **Send the FULL knob dict every tick.** `sendParams(raw, playbackPos)`
   at UI rate (the demo uses ~125 Hz), `playbackPos` in **seconds**
   (`player.positionSec`). The server merges and clamps server-side; you
   do not need client-side validation to be safe, only to be polite.
   **Seed `raw` from your session's live state, NOT from the manifest's
   static defaults.** Manifest defaults can differ from what your session
   was configured with — e.g. blindly asserting `steps_override`'s
   default of 8 against a session started with `steps: 4` forces a
   pipeline rebuild on tick 1. For knobs your UI doesn't control, either
   omit them or initialize them from the session config you sent /
   the `params_update` echo, then hold those values.
2. **Never skip, reorder, or drop slices on the floor.** Delta-flagged
   slices are zstd deltas against your local buffer mirror; every slice
   must reach `player.patch` / `player.addDelta` or the basis desyncs and
   audio corrupts from then on.
3. **Respect the epoch guard.** In your `slice` listener, drop any slice
   with `slice.epoch !== player.swapCount`. In your `swap_ready`
   listener, call `player.swap(detail.interleaved, detail.channels)` —
   that bumps `swapCount` in lockstep with the epoch the backend just
   bumped. Stale in-flight slices from the old track then drop instead of
   bleeding through.
4. **Serve the worklet.** Copy `assets/audio-worklet.js` into your static
   dir and point `AudioPlayerOptions.workletUrl` at it (default
   `/audio-worklet.js?v=5`). Without it, playback falls back to
   ScriptProcessor (works, but worse).
5. **Reconnect = full re-handshake.** There is no session resume. On an
   unexpected close (`closedByUser === false`), build a new
   `RemoteBackend` from your retained config/PCM, attach ALL listeners
   **before** `connect()`, and call
   `remote.setSliceEpoch(player.swapCount)` so the fresh epoch counter
   matches the surviving player. `wsReconnect.ts` provides the backoff
   loop; the demo's `demos/realtime_motion_graph_web/web/hooks/useStartSession.ts` is the reference
   orchestration.

## Echo channels (external control)

Exactly two commands are `origin_sensitive` in the contract: `params`
and `set_prompt_blend`. Sent on your own WS they apply directly; sent
externally (MCP / control bus) they are NOT applied — the server echoes
them back on the command's `echo_event` (`params_echo`,
`prompt_blend_echo`) for YOUR UI to mirror into its own state and
re-send. Every other command applies identically from any origin and
acks via its normal events (`lora_catalog`, `timbre_set`, `swap_ready`,
...). If you want agent-driven control to move your sliders, subscribe
to the echo events (the demo's `useMcpMirror.ts` is the reference).

## State management

The SDK never writes your stores. Subscribe to `RemoteBackend`'s
CustomEvents (`ready`, `slice`, `params`, `params_echo`, `swap_ready`,
`swap_failed`, `stem_assets`, `depth_applied`, `lora_catalog`, `close`,
...) and mirror what you need. Inject app behavior through the option
seams: `RemoteBackendOptions.promptTransform` (e.g. LoRA trigger
prefixing) and `AudioPlayerOptions.loudnessConfig` / `workletUrl`.

## Operator defaults (`config.json`) — client-side, not a manifest

Separate from the two backend manifests above: `config.json` is a
hand-authored, per-installation file of *startup defaults* (prompts, knob
start values, enabled LoRAs, engine/session fields, plus per-client UI
blocks). The portable schema + pure transforms live in `config/` and are
shared across frontends; import them from the package root:

- `loadConfig(baseUrl?)` → merge `config.json` onto `DEFAULT_CONFIG`.
- `rtmgConfigToSessionConfig(cfg, runtime)` → the config → handshake
  `SessionConfig` mapping. Use it to start your session identically to
  every other frontend.
- `applyConfigToState` / `captureConfigFromState` → neutral apply/capture
  adapters; write a thin wrapper from your own state to the snapshot shape
  (the SDK never touches your store).
- `serializeConfig` → **preserve-unknown on write**: re-emit top-level keys
  you read but don't model, so a config another client authored isn't
  silently truncated when you export.

Unlike the manifests, this file is hand-edited (comment-heavy `_comment` /
`_*_help` keys): read ignores unknown keys, write preserves them, and a
`version` field is present from day one.

## Control copy (`controls/`) — user-facing, also not a manifest

The descriptions and display names you render ON the controls. Two layers,
don't conflate them: the `/api/knobs` manifest's `description` is the **terse,
agent-facing** one-liner (registry-generated); `controls/` is the **editorial,
when-to-reach-for-it** prose (hand-authored, client-side). Import from the
package root:

- `describeControl(param)` → rich description or `undefined` (LoRA / manual
  slots share one string each, matched by prefix).
- `displayNameFor(param)` → friendly label (`denoise` → `strength`, else
  snake_case → spaced).
- `resolveControlDescription(param, manifest?)` → prefer rich copy, fall back
  to the manifest's `description` so runtime-only knobs still read something.

Also here: `TERMS` — the canonical product nouns that get renamed (`tags`,
`lora`/`lora_plural`). Stable id, plain data, exported from `node.ts` too, so
the in-progress native (VST) codegen emits labels from the same source.
Compose compound labels by interpolating (`` `${TERMS.lora} Library` ``);
flip the noun once and web + VST follow. Headline labels use it; deep inline
sentences keep their literals (term-fragmenting prose is churn, not a win).

The tooltip-rendering machinery and web-only affordance hints (keyboard
chords, MIDI-learn) stay in your client — only the copy is shared.

## Reference implementations in the demo app

- `demos/realtime_motion_graph_web/web/components/Performance/DynamicKnobPanel.tsx` — a complete control
  surface rendered purely from `/api/knobs`, zero hand-declared knobs.
  This is the pattern to copy for any new UI.
- `demos/realtime_motion_graph_web/web/hooks/useStartSession.ts` — session + reconnect orchestration.
- `demos/realtime_motion_graph_web/web/hooks/useMcpMirror.ts` — echo-channel mirroring.

## If you edit the SDK itself

- No imports from the host app: everything internal is relative; the
  only allowed bare import is `fzstd` (`tests/unit/test_client_sdk.py`
  enforces this).
- After any change to the Python registries (`protocol.py`,
  `acestep/streaming/knobs.py`), regenerate:
  `python demos/realtime_motion_graph_web/scripts/gen_wire_types.py` —
  a stale `types/wireContract.gen.ts` fails CI.
- `assets/audio-worklet.js` and the demo's `public/audio-worklet.js`
  must stay byte-identical (drift-tested). Edit one, copy to the other.
- Binary framing (the `<II` PCM header, the 23-byte slice header) is
  hand-coded by design; it lives in `packPcmFrame` / the slice decoder
  and is documented per-entry in the registry, not typed.
- `config/` is the operator-defaults `config.json` schema + pure transforms
  (the one hand-authored, client-side contract — not generated). Keep it
  pure: no store/DOM imports cross into it, so M4L / VST can consume it.
- `controls/` is the user-facing knob/input copy + the `TERMS` product-noun
  lexicon (the second hand-authored, client-side layer — editorial prose +
  display labels, not the registry's terse `description`). Pure data + pure
  functions, no store/DOM imports, so the native (VST) codegen can read it.
- If you change the SDK's bundled surface (`controls/`, `config/`, etc.),
  rebuild so the committed `dist/` static-demo bundles don't go stale:
  `cd packages/demon-client && npm run build`.

## Agent-driven control without a browser

The MCP server exposes the same contract: `describe_protocol` and
`list_knobs` return the manifests; `set_knob(s)` / `set_prompt` /
`swap_to_fixture` / etc. drive a live session and are validated against
the same registries the wire is.
