# Game Asset Motion Dogfood

Date: 2026-05-20

## Context

This dogfood explored whether vlmkit can move beyond static generated images
into game asset production:

`image/model reference -> asset contract -> GLB/OBJ -> motion IR -> retargeted GLB -> render/quality gate`

The concrete scenario used a voxel robot and real `.vrma` motion samples from
`tk256ailab/vrm-viewer`, kept under ignored `design-runs/.../external/`.

## What Works

- `gpt-image-2` is useful for concept and reference sheets, but not direct 3D
  output. The useful boundary is image reference -> deterministic asset
  contract -> generated or modeled GLB.
- Fixed-camera Three.js + Playwright renders are good enough as a cheap,
  repeatable visual smoke gate for GLB/OBJ and animation clips.
- VRMA is already close to the desired adapter boundary when it is binary glTF
  with `VRMC_vrm_animation.humanoid.humanBones`.
- A small Motion IR is effective. It decouples input adapters from target model
  generation:
  `VRMA/Mixamo/etc -> Motion IR -> retarget -> GLB -> render gate`.
- Real `.vrma` samples `LookAround`, `Goodbye`, and `Jump` all pass the
  mechanical pipeline:
  fetch, extract, verify IR, apply to robot, verify GLB, render, verify renders.
- The same three samples now pass the deterministic quality gate when the target
  is declared as a simplified rig:
  `--root-translation-mode relative --retarget-profile robot-voxel`.
- `robot-voxel` is now a named weighted retarget profile. `simple-rig` remains
  an alias, but the report records the canonical profile name, score, weighted
  penalty, and skipped-by-policy breakdown.
- Smoke runs can require `--min-quality pass` so warnings do not silently pass
  CI-style loops. The failure report keeps the quality summary for debugging.
- `apply-motion-ir --audit-out` now writes a root normalization audit. The
  smoke report summarizes source root height, target base height, root delta
  ranges, normalized ranges, applied scale, and a root scaling recommendation
  for each root translation track.
- The same audit now records target rig bind metrics from retargeted node world
  positions: skeleton bounds, skeleton height, hand span, foot spread, and
  pelvis-to-foot height.
- Motion IR extraction now records source humanoid rest metrics from VRMA
  nodes. `apply-motion-ir` compares those source metrics with target bind
  metrics and emits an initial pose warning.
- Animation renders wait for two extra browser frames after viewer readiness
  before screenshot capture, reducing first-frame WebGL paint races.
- The renderer saves canvas PNG data directly and retries transparent capture
  buffers by reloading the viewer once. This is more stable than page-level
  screenshots for first-frame WebGL smoke tests.
- `ground-y` is now checked with `groundDeltaY` when render metadata includes
  normalized bind bounds, avoiding false warnings from camera-fit world
  coordinates.
- The animation viewer records tracked pelvis, hand, and foot positions.
  `verify-motion-quality` now emits a `foot-contact` check based on
  `left_foot` / `right_foot` bind-vs-animated delta.
  It also emits a `limb-extent` check from tracked node displacement.
- `verify-motion-quality-gold` now compares smoke reports against a committed
  small calibration fixture for `LookAround`, `Goodbye`, and `Jump`.
- `compare-motion-quality-reports` now compares candidate smoke reports against
  a baseline. In the current Jump test, `scale-to-model` improves ground error
  but regresses foot contact and pelvis displacement, so the decision is
  `candidate-tradeoff`.
- Smoke reports now expose normalization candidates. The candidate planner
  turns runnable candidates into smoke/compare commands, including the first
  runnable pose pre-normalization candidate.
- `--pose-normalization arm-rest-offset` now computes source-to-target
  upper/lower arm rest offsets and records the applied offsets in the audit.
  `LookAround` stayed `stable`, `Goodbye` improved one displacement metric,
  and `Jump` regressed it, so this is a runnable experiment path rather than a
  new default.
- `--pose-normalization stance-width-offset` now computes lower-body rest
  offsets for stance mismatch candidates. The dogfood run caught a lateral-axis
  sign problem: naive world-vector alignment produced 116deg upper-leg offsets
  and worse foot contact. Matching the source lateral sign to the target side
  reduced the offset to about 3.5deg, and the `Goodbye` candidate now compares
  as stable.
- Arm-rest candidates now have a motion-aware gate. The audit measures max
  upper-arm rotation and only emits a runnable arm-rest candidate at 60deg or
  more. In this set, `Goodbye` passes with 108.4deg while `LookAround` 14.5deg
  and `Jump` 30.0deg become `needs-motion-evidence`.
- `select-motion-normalization-candidates` now groups runnable candidate
  comparisons and emits a selection report. After motion gating, it reports
  `arm-rest-pose-offset` as `needs-policy` and keeps `root-scale-to-model`
  rejected. Non-automatic candidates now require at least 3 compared samples,
  at least 2 improvements, and no regressions/tradeoffs/missing comparisons
  before they become `promotable`.
- A first MoonBit `motion-core` package now ports the pure policy slice:
  arm-rest motion gate status, arm-rest candidate status, root-translation
  recommendation/candidate selection, pose mismatch warning-id selection, and
  pose normalization candidate spec selection, plus group-level candidate
  selection. A thin `motion-core-cli` entrypoint builds to JS and preserves the
  same command decisions under both `moon run` and direct `node` execution.
  `check-motion-core-parity` pins the generated JS command decisions for the
  current fixtures. `motion-core-runtime` now lets the JS orchestration scripts
  call those generated decisions while JS still performs GLB/JSON orchestration
  and audit detail mapping. The JS adapter exposes categorized
  `motionCorePolicy.root`, `.pose`, and `.selection` APIs to keep policy
  ownership explicit.
- The dry-run VLM review scaffold can create a contact sheet and strict JSON
  prompt for cheap reviewers such as UI-TARS / Nova Lite without adding a
  blocking human step.

## What Failed Or Warned

Before the latest loop, all three tested external VRMA samples produced the
same quality warnings:

- retained tracks: 16
- skipped channels: 36
- skipped regions: 30 finger, 2 body, 2 arm, 2 toe/foot
- quality verdict: `warn`
- warning checks: `ground-y`, `retained-channels`

After adding relative root translation, normalized ground deltas, and the
`robot-voxel` retarget profile, the same batch reports:

- quality verdict: `pass`
- retarget profile score: 1.0
- weighted penalty: 0
- skipped-by-policy: 30 finger ignored, 4 upper-body fallback, 2 toe ignored
- source initial root height: `LookAround` 0.87944, `Goodbye` 0.89459,
  `Jump` 0.86145
- target base root height: 1.25
- root delta y range: `LookAround` -0.006..0.003, `Goodbye` -0.007..0.011,
  `Jump` -0.094..0.230
- `LookAround` groundDeltaY: -0.047..-0.012
- `Goodbye` groundDeltaY: -0.031..-0.013
- `Jump` groundDeltaY: -0.107..-0.052
- `LookAround` foot minDeltaY: -0.009..0.009
- `Goodbye` foot minDeltaY: -0.004..0.007
- `Jump` foot minDeltaY: 0.004..0.118
- tracked node max displacement: `LookAround` 0.991, `Goodbye` 1.012,
  `Jump` 1.104; pelvis stays under 0.183 across the batch
- retained-channel ratio: 0.3077, accepted because only simplified-rig
  tolerated regions are skipped

This means parser correctness is no longer the main blocker. The next blocker
is adding more target rigs and expanding the gold set before promoting the
schema out of the dogfood directory.

## Product Lessons

- The main loop should not depend on human visual review. Human input should
  calibrate thresholds and gold fixtures; the working loop should emit
  machine-readable `pass` / `warn` / `fail`.
- VLM reviewers should be second opinions over deterministic evidence, not the
  source of truth. They should receive a contact sheet plus metrics and return
  strict JSON.
- `warn` is useful. It let the agent continue while preserving the exact
  quality debt. In this run, `warn` clearly identified that the ground metric
  was measuring the wrong coordinate space and that skeleton coverage needed a
  target-profile policy.
- Skip counts alone are too coarse. Region-level skip aggregation is necessary
  because dropping 30 finger channels is less severe for a blocky robot than
  dropping hips, head, or upper arms.
- Root translation must be policy-controlled. For generated simplified
  characters, `relative` is safer than copying source hips coordinates because
  it keeps the target bind height and only applies motion deltas.
- Normalization must leave a machine-readable audit trail. Otherwise an agent
  cannot tell whether a bad render came from parser drift, root scaling, or the
  target rig's bind pose.
- The audit should recommend, not silently switch, root scaling policy. In the
  current three-sample set, `LookAround` and `Goodbye` stay `relative-ok`
  because their vertical root motion is tiny; `Jump` emits
  `consider-scale-to-model` because source/target root height differs and the
  clip has meaningful vertical root motion.
- A recommendation needs a paired comparison run. `scale-to-model` passed for
  Jump, but it was not strictly better: ground error improved while foot contact
  and pelvis displacement regressed. That argues for keeping `relative` as the
  smoke default until the target bind-height measurement is richer.
- `--fail-on-tradeoff` is important for autonomy. A candidate can pass quality
  gates and still be mixed; the plan runner must not treat that as an automatic
  replacement for the baseline.
- Target bind metrics are now available in the audit. For the voxel robot the
  measured retarget skeleton is height 1.88, hand span 1.12, foot spread 0.48,
  and pelvis-to-lowest-foot height 1.02. This is the right input for source
  rest-pose comparison; `targetBaseRootHeight` alone was only pelvis height.
- Source rest metrics show why this should be pose-aware rather than only
  scale-aware. The VRMA source skeleton height is 1.304, hand span 0.926, and
  foot spread 0.140. Core scale ratios are close enough to be usable
  (`skeletonHeight` 1.442, `legHeight` 1.258, `handSpan` 1.209), but foot
  spread scale is 3.438, so the audit emits `foot-spread-mismatch`.
- Pose mismatch scoring now also measures shoulder width, upper-leg spread,
  and arm rest angle. The current source is effectively T-pose
  (`armDownAngleDeg` 90) while the voxel target arms hang down
  (`armDownAngleDeg` 1.3), so the audit emits `arm-rest-angle-mismatch`.
  Shoulder width and upper-leg spread also diverge from skeleton-height scale.
- The first arm-rest pre-normalization is mechanically runnable but only useful
  when the clip actually exercises upper arms. Gating on upper-arm rotation
  removes the known regressing `Jump` case while keeping the improved
  `Goodbye` case visible for policy work.
- Stance-width pre-normalization is also mechanically runnable, but it should
  stay non-automatic until multiple samples show improvement. The current
  value is exposing safe candidate execution and catching lateral-axis mistakes
  without promoting a default.
- A selection layer is now necessary between "candidate is runnable" and
  "candidate should change defaults." Per-sample comparisons are too granular
  for autonomous loops; the group-level report gives the agent one stable
  contract to reject mixed evidence, hold weak evidence as `needs-policy`, or
  mark repeated safe improvements as `promotable`.

## Next Implementation Order

1. **Retarget profile policy v2**
   - Add more named profiles beyond `robot-voxel`.
   - Calibrate the new machine-checkable profile schema against more targets.
   - Calibrate weighted region scoring against more targets.
   - Example: dropped fingers = pass/low cost, dropped hips/head/upper arms =
     fail.

2. **Root and ground normalization**
   - Use the new report comparison output to calibrate when
     `consider-scale-to-model` should escalate from warning to candidate mode.
   - Decide whether pose warnings should trigger an automatic pre-normalization
     pass, starting with T-pose/A-pose to arms-down target rigs.
   - Add source/target up-axis assumptions once a non-Y-up source appears.
   - Add actionable suggestions when `groundDeltaY` exceeds thresholds.

3. **Motion quality gate v2**
   - Expand the current three-sample gold set with more styles and target rigs.
   - Include per-clip and per-region scores in the JSON report.

4. **VLM review real run**
   - Run UI-TARS first because it is cheap and fast in the existing VLM bench.
   - Use Nova Lite as stable fallback / second reviewer.
   - Measure whether VLM adds signal beyond deterministic warnings.

5. **Kagura smoke**
   - Pre-runtime handoff smoke now exists: validate GLB path, clip ids,
     scale/origin, axes, fixed camera views, and snapshot verification with
     `motion:kagura-handoff:game-assets`.
   - Runtime probe now exists: `run-kagura-runtime-smoke.mjs` loads the GLB via
     local `mizchi/kagura` `gltf_viewer` and captures the canvas with
     Playwright. The first robot run reaches canvas load but fails on a black
     frame (`visiblePixelRatio=0.0111`) plus WebGPU invalid texture/command
     buffer warnings. Kagura's own `test_scene.glb` fails the same way, so this
     is a headless Kagura/WebGPU smoke blocker rather than a generated-asset
     blocker. The probe now separates frame-signal from frame-substance checks:
     a render submit can be observed even while the visible frame remains black.
     `--calibration-contract` records a known-good Kagura asset in the same
     report and sets `environmentLikelyBroken` when both paths fail.
   - Next step is to fix/calibrate the Kagura headless render path and expose
     real clip playback status, then emit load status, clip status, and frame
     status in the same structured JSON style.

6. **Promote stable tools**
   - After the retarget and normalization policy stabilizes, move Motion IR,
     retarget profile validation, and quality gates out of `design-runs`.

## Current Commands

```bash
node design-runs/game-assets-20260520/tools/run-external-vrma-smoke.mjs \
  --samples LookAround,Goodbye,Jump \
  --root-translation-mode relative \
  --retarget-profile robot-voxel \
  --min-quality pass \
  --review-vlm \
  --review-dry-run

node design-runs/game-assets-20260520/tools/verify-motion-quality-gold.mjs \
  --report design-runs/game-assets-20260520/external/vrma/tk256ailab/smoke-report.json \
  --gold design-runs/game-assets-20260520/motions/external-vrma-quality-gold.json

node design-runs/game-assets-20260520/tools/compare-motion-quality-reports.mjs \
  --baseline design-runs/game-assets-20260520/external/vrma/tk256ailab/smoke-report.json \
  --candidate design-runs/game-assets-20260520/external/vrma/tk256ailab/smoke-report.scale-to-model.json \
  --samples Jump \
  --fail-on-regression \
  --fail-on-tradeoff

node design-runs/game-assets-20260520/tools/plan-motion-normalization-candidates.mjs \
  --report design-runs/game-assets-20260520/external/vrma/tk256ailab/smoke-report.json
```

Useful ignored outputs:

- `design-runs/game-assets-20260520/external/vrma/tk256ailab/smoke-report.json`
- `design-runs/game-assets-20260520/external/vrma/tk256ailab/*.motion-quality.json`
- `design-runs/game-assets-20260520/external/vrma/tk256ailab/*.motion-quality.contact-sheet.png`
- `design-runs/game-assets-20260520/external/vrma/tk256ailab/*.vlm-review.json`

## Sources

- https://note.com/npaka/n/nde5589d13536
- https://vrm.dev/en/vrma/
- https://github.com/vrm-c/vrm-specification/tree/master/specification/VRMC_vrm_animation-1.0
- https://github.com/tk256ailab/vrm-viewer
- https://openrouter.ai/bytedance/ui-tars-1.5-7b
- https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-amazon-nova-lite.html
