ITADN
simonw/llm/Issues

Plugin hook for pre-execution tool/command trust verification

#1461Openvdineshk 创建于 2026-05-26
V
vdineshkcommented
## Problem `llm` supports plugins that can register new models, templates, and tools. As the plugin ecosystem grows — and especially as tool-use and agent workflows become more common — there is no hook that lets a user or plugin verify whether a tool endpoint (e.g., an MCP server) is trustworthy *before* `llm` dispatches a call to it. Today: - Any installed plugin's tools are trusted implicitly once the plugin is installed. - There is no middleware point where an external reputation check, allowlist lookup, or conformance verification can run before tool execution. - Users running `llm` in automated pipelines have no programmatic way to gate tool calls based on third-party trust signals. ## Proposal Add a **pluggable pre-execution hook** to the tool/command dispatch path — similar to how `register_commands` or `register_models` work, but for gating execution: ```python import llm @llm.hookimpl def before_tool_execution(tool_name: str, server_uri: str, parameters: dict) -> bool: """ Return True to allow execution, False to block. Raise an exception to abort with a message. Called before every tool invocation. """ ... ``` Key design points: 1. **Runs before dispatch** — not after. The hook can prevent a call from ever being made. 2. **Pluggable via pluggy** — any plugin can register a `before_tool_execution` hook. Multiple hooks can coexist (all must pass). 3. **Async-safe** — the hook should support calling an external API (reputation service, policy engine, local DB) without blocking. 4. **Fail-closed default** — if the hook raises or times out, the tool call is denied. ## Use cases - A security-conscious user installs a plugin that checks tool endpoints against a behavioral trust score API before allowing execution. - An enterprise deployment blocks any tool server not on a corporate allowlist. - An audit plugin logs every tool invocation decision (allowed/blocked) for compliance. ## Alternatives considered - **Plugin allowlist in config** — could work for static lists, but doesn't support dynamic or score-based decisions. - **Post-execution logging only** — too late to prevent unwanted side-effects or data exfiltration. Happy to help draft an implementation if this direction makes sense.
0 条评论