ITADN

[P0 data-corruption] Simulation.reset() does not flush dataset episode buffer → multi-episode runs collapse to 1 episode

#710Closedcagataycali 创建于 2026-06-26
bug
C
cagataycalicommented
## Summary `Simulation.reset()` does **not** flush the dataset recorder's episode buffer. When an agent (or any caller) records a dataset across multiple `reset → run_policy` cycles, all frames from N cycles concatenate into **one giant episode** with `total_episodes=1`. ## Repro `molmoact-e2e-2026-06-25/e2e_agent_test.py` runs the natural-language prompt: > "Run the allenai/MolmoAct2-SO100_101 policy for 20 episodes with 60 steps per episode..." The agent correctly emits the structure (forensics from `agent_messages.json` of run `20260626-150038`): ``` [0] so101_sim: {action: 'reset'} [2] so101_sim: {action: 'run_policy', n_steps: 60, instruction: 'pick up the red cube', ...} [4] so101_sim: {action: 'reset'} [6] so101_sim: {action: 'run_policy', n_steps: 60, ...} ... (10 reset+run_policy pairs total) ... [32] so101_sim: {action: 'stop_recording'} [34] so101_sim: {action: 'render', ...} [36] so101_sim: {action: 'destroy'} ``` Dataset result (`runs/20260626-150038/dataset/meta/info.json`): ```json {"total_episodes": 1, "total_frames": 1140} ``` Expected: `total_episodes=10, total_frames=600` (10 × 60). Got `1 × 1140` (one continuous concatenation; ~114 frames per cycle because n_substeps yields ~1.9 frames/step). **Reproduced identically in `runs/20260626-140044`** — same fingerprint. This is a stable repro, not flaky. ## Root cause `strands_robots/simulation/mujoco/simulation.py:1419`: ```python def reset(self) -> dict[str, Any]: # ... with self._lock: mj.mj_resetData(self._world._model, self._world._data) self._world.sim_time = 0.0 self._world.step_count = 0 for r in self._world.robots.values(): r.policy_running = False r.policy_steps = 0 return {"status": "success", "content": [{"text": "Reset to initial state."}]} ``` There is no `recorder.save_episode()` call. Comment at line 2387 confirms the design assumption: *"the caller will save_episode them"* — but no caller does. The Robot tool router does not expose `save_episode` as an action either, so the LLM has no path to emit it even if it knew to. ## Impact - **P0 silent data corruption**: every multi-episode MolmoAct/π/SmolVLA/ACT eval run that uses `reset + run_policy + record` produces a dataset that lies about episode count. - **MolmoAct2 has been wrongly suspected** of "not moving / not segmenting" for ~2 days — it actually ran the requested 10 episodes; the dataset just collapsed them. - Any downstream training/eval that filters by `episode_index` gets garbage. - Affects all sim backends? Need to check `newton/simulation.py` reset() too. ## Recommended fix (option A — auto-flush on reset) Inject before `mj_resetData`: ```python def reset(self) -> dict[str, Any]: if self._world is None or self._world._model is None or self._world._data is None: return {"status": "error", "content": [{"text": _NO_WORLD_MSG}]} if err := self._require_no_running_policy("reset"): return err # NEW: episode boundary — flush buffered frames as a complete episode # before clearing physics state. if self._world._backend_state.get("recording") and self._recorder is not None: if self._recorder.has_buffered_frames(): # add this helper self._recorder.save_episode() mj = self._mj with self._lock: mj.mj_resetData(...) ... ``` Rationale: "reset = new episode" is the LeRobot semantic users expect; explicit `save_episode` actions would require LLM prompt changes and are brittle. Auto-flush is backward compatible (no recording → no-op; no buffered frames → no-op). ## Alternative (option B): expose `save_episode` as a router action Less invasive to existing reset() but requires updating `e2e_agent_test.py` prompt to instruct the agent: *"After each run_policy, before the next reset, call save_episode."* Trusts LLM compliance — not great for replayable benches. ## Acceptance criteria - [ ] `reset()` auto-flushes buffered frames as a sealed episode when recording is active - [ ] Add `has_buffered_frames()` helper to `DatasetRecorder` (or use existing `len(frames) > 0` accessor) - [ ] Unit test: `reset()` between two `run_policy(60)` calls produces 2 episodes in `info.json` - [ ] Re-run `molmoact-e2e-2026-06-25` end-to-end → expect `total_episodes=10` (or 20 if the secondary agent-iteration-budget issue is also fixed) - [ ] Audit `newton/simulation.py` reset() for same bug - [ ] Add CHANGELOG entry: silent data-corruption fix ## Secondary observation (separate ticket) The agent did **10** reset/run_policy pairs, not 20 (prompt asked for 20). Likely max_iterations / token budget cap on the agent loop. File separately once the dataset bug is fixed, since we can't measure the true episode count until then. ## Evidence files - `~/molmoact-e2e-2026-06-25/runs/20260626-150038/agent_messages.json` (39 messages, 19 tool calls, full pattern) - `~/molmoact-e2e-2026-06-25/runs/20260626-150038/dataset/meta/info.json` (total_episodes=1, total_frames=1140) - `~/molmoact-e2e-2026-06-25/runs/20260626-140044/` (identical pattern, prior day)
关闭于 2026-06-26 5 条评论