Plugin hook for pre-execution tool/command trust verification
## Problem
`llm` supports plugins that can register new models, templates, and tools. As the plugin ecosystem grows — and especially as tool-use and agent workflows become more common — there is no hook that lets a user or plugin verify whether a tool endpoint (e.g., an MCP server) is trustworthy *before* `llm` dispatches a call to it.
Today:
- Any installed plugin's tools are trusted implicitly once the plugin is installed.
- There is no middleware point where an external reputation check, allowlist lookup, or conformance verification can run before tool execution.
- Users running `llm` in automated pipelines have no programmatic way to gate tool calls based on third-party trust signals.
## Proposal
Add a **pluggable pre-execution hook** to the tool/command dispatch path — similar to how `register_commands` or `register_models` work, but for gating execution:
```python
import llm
@llm.hookimpl
def before_tool_execution(tool_name: str, server_uri: str, parameters: dict) -> bool:
"""
Return True to allow execution, False to block.
Raise an exception to abort with a message.
Called before every tool invocation.
"""
...
```
Key design points:
1. **Runs before dispatch** — not after. The hook can prevent a call from ever being made.
2. **Pluggable via pluggy** — any plugin can register a `before_tool_execution` hook. Multiple hooks can coexist (all must pass).
3. **Async-safe** — the hook should support calling an external API (reputation service, policy engine, local DB) without blocking.
4. **Fail-closed default** — if the hook raises or times out, the tool call is denied.
## Use cases
- A security-conscious user installs a plugin that checks tool endpoints against a behavioral trust score API before allowing execution.
- An enterprise deployment blocks any tool server not on a corporate allowlist.
- An audit plugin logs every tool invocation decision (allowed/blocked) for compliance.
## Alternatives considered
- **Plugin allowlist in config** — could work for static lists, but doesn't support dynamic or score-based decisions.
- **Post-execution logging only** — too late to prevent unwanted side-effects or data exfiltration.
Happy to help draft an implementation if this direction makes sense.
0 条评论