[BUG] Monte Carlo layer uses nonexistent sdk.stream() instead of sdk.query()
### Preliminary Checks
- [x] I have read the [Code of Conduct](https://github.com/wshobson/agents/blob/main/.github/CODE_OF_CONDUCT.md)
- [x] I have searched existing issues to ensure this is not a duplicate
- [x] This report contains only technical information about a bug
### Affected Subagent
plugin-eval (Layer 3: monte_carlo)
### Bug Description
`run_simulation()` in `src/plugin_eval/layers/monte_carlo.py` calls `sdk.stream()`, which does not exist in `claude-agent-sdk`. The judge layer correctly uses `sdk.query()` with `ClaudeAgentOptions` and `ResultMessage`, but the monte carlo layer references a nonexistent API.
Because the call is wrapped in a bare `except`, every simulation silently fails with `errored=True`. This causes Layer 3 to report 100% failure rate, which tanks the composite score on dimensions like robustness, triggering accuracy, and token efficiency.
### Steps to Reproduce
1. Install with LLM extras: `uv run --extra llm plugin-eval certify <any-skill-path>`
2. Observe the monte_carlo layer score is artificially low
3. Add logging in the `except` block to confirm: `AttributeError: module 'claude_agent_sdk' has no attribute 'stream'`
### Expected Behavior
Monte Carlo simulations should run successfully using `sdk.query()`, the same pattern the judge layer uses in `layers/judge.py`.
### Additional Context
The fix is replacing the `sdk.stream()` block in `run_simulation()` with `query()` + `ClaudeAgentOptions` + `ResultMessage`, matching the working pattern in `judge.py`. PR at https://github.com/wshobson/agents/pull/478
关闭于 2026-04-16 0 条评论