ITADN

[BUG] Monte Carlo layer uses nonexistent sdk.stream() instead of sdk.query()

#477Closedslgoodrich 创建于 2026-04-10
S
slgoodrichcommented
### Preliminary Checks - [x] I have read the [Code of Conduct](https://github.com/wshobson/agents/blob/main/.github/CODE_OF_CONDUCT.md) - [x] I have searched existing issues to ensure this is not a duplicate - [x] This report contains only technical information about a bug ### Affected Subagent plugin-eval (Layer 3: monte_carlo) ### Bug Description `run_simulation()` in `src/plugin_eval/layers/monte_carlo.py` calls `sdk.stream()`, which does not exist in `claude-agent-sdk`. The judge layer correctly uses `sdk.query()` with `ClaudeAgentOptions` and `ResultMessage`, but the monte carlo layer references a nonexistent API. Because the call is wrapped in a bare `except`, every simulation silently fails with `errored=True`. This causes Layer 3 to report 100% failure rate, which tanks the composite score on dimensions like robustness, triggering accuracy, and token efficiency. ### Steps to Reproduce 1. Install with LLM extras: `uv run --extra llm plugin-eval certify <any-skill-path>` 2. Observe the monte_carlo layer score is artificially low 3. Add logging in the `except` block to confirm: `AttributeError: module 'claude_agent_sdk' has no attribute 'stream'` ### Expected Behavior Monte Carlo simulations should run successfully using `sdk.query()`, the same pattern the judge layer uses in `layers/judge.py`. ### Additional Context The fix is replacing the `sdk.stream()` block in `run_simulation()` with `query()` + `ClaudeAgentOptions` + `ResultMessage`, matching the working pattern in `judge.py`. PR at https://github.com/wshobson/agents/pull/478
关闭于 2026-04-16 0 条评论