Agent integration via experimental.kernels — MVP shipped, two upstream warts, broader pair-programming design

## Use case: external agent driving the editor's kernel via experimental.kernels External agents (Claude Code CLI, Codex, MCP servers, sibling extensions) increasingly want to drive a marimo notebook **in the same kernel the user sees in the editor** — not a sidecar `marimo edit` browser kernel. The `experimental.kernels` API on `marimo-team.vscode-marimo`'s `activate()` return value is exactly the integration point for that (the vscode-jupyter-shaped one mentioned in #545). It works today. I built [`hugolytics/agent-bridge`](https://github.com/hugolytics/agent-bridge) on top of it as a tight existence proof — a ~250 LOC VS Code extension that hosts a localhost HTTP server inside VS Code, maps marimo-pair's SSE protocol onto `experimental.kernels.executeCode`, and exposes notebook cell CRUD via `WorkspaceEdit`. The unmodified [`marimo-pair`](https://github.com/marimo-team/marimo-pair) skill works against it (just `--url http://127.0.0.1:<port>`); 22 of 24 capabilities I drew up pass. Filing this per maintainer guidance on #545: design discussion before public API changes. ## Two upstream warts blocking the MVP These both reproduce against `marimo-team.vscode-marimo@0.13.1` from the marketplace, no fork involved. Each ships with a small targeted PR (RED→GREEN against `upstream/main`). I'm happy to revise the implementation choice per maintainer preference — what matters is grounding the discussion in a verifiable repro + a verifiable fix. ### Wart B — `marimo._code_mode.get_context()` raises from scratchpad **Repro** (anywhere `experimental.kernels.executeCode` is called or `marimo.api` `execute-scratchpad` is invoked): ```python import marimo._code_mode as cm async with cm.get_context() as ctx: pass # → RuntimeError: NotebookDocument not available — code_mode must be invoked # via the /api/execute endpoint which sets the document context variable ``` **Root cause:** `handle_execute_scratchpad` in `marimo._runtime.runtime` populates the `notebook_document_context` ContextVar from `request.notebook_cells`. `execute_scratch` in `marimo_lsp/api.py` currently sends `ExecuteScratchpadCommand(code=args.inner.code)` with no `notebook_cells`, so the ContextVar stays None. **Fix approaches considered:** | | Where | Pros | Cons | |---|---|---|---| | **B1 (proposed)** | `marimo_lsp/api.py` — pass `notebook_cells=tuple(session.app_file_manager.app.cell_manager.document.cells)` on send | 5 LOC. Mirrors `marimo._server/api/endpoints/execution.py:319` literally. Uses the public `cell_manager.document` accessor. | O(n) document snapshot per scratchpad call (negligible for typical notebook size; could be measured if you want). | | B2 | `marimo._runtime.runtime` — have `handle_execute_scratchpad` fall back to a kernel-internal document when `notebook_cells` is None | Callers don't need to remember. | Cross-repo (marimo not marimo-lsp); larger blast radius. | | B3 | `marimo._runtime.commands` — accept a lazy `notebook_cells` callable so snapshot happens only if `code_mode` is actually used | Avoids snapshot cost when unused. | API change in marimo; broader surface. | **Proposed PR:** [`hugolytics:fix/code-mode-scratchpad-context`](https://github.com/hugolytics/marimo-lsp/tree/fix/code-mode-scratchpad-context) (B1). Adds a pytest (`test_scratchpad_code_mode_get_context`) that runs scratchpad code calling `cm.get_context()` and asserts no `marimo-error` arrives and stdout reports the cell count. RED on `main`, GREEN with the patch. ### Wart C — scratchpad exceptions arrive as `application/vnd.code.notebook.stderr`, not `.error` **Repro:** ```ts const ext = vscode.extensions.getExtension("marimo-team.vscode-marimo"); const api = await ext.activate(); const kernel = await api.experimental.kernels.getKernel(notebookUri); for await (const output of kernel.executeCode("1/0")) { for (const item of output.items) { console.log(item.mime, new TextDecoder().decode(item.data)); } } // → application/vnd.code.notebook.stderr ZeroDivisionError: division by zero // → application/vnd.code.notebook.stdout <text/html traceback> // (no application/vnd.code.notebook.error item ever arrives) ``` **Root cause:** `extension/src/kernel/ExecutionRegistry.ts` line 675 (`buildOutputItem`) translates `marimo-error` channel data to `NotebookCellOutputItem.stderr(...)` rather than `.error(...)`. The wrapping `NotebookCellOutput` has `{channel: "marimo-error"}` metadata, but its items don't, so consumers iterating the `AsyncIterable<Output>` from `experimental.kernels.executeCode` can't distinguish a failed run from a successful run that printed to stderr. (vscode-jupyter — the API this is modeled after — uses the `.error` mime for exactly this signal.) **Fix approaches considered:** | | Where | Pros | Cons | |---|---|---|---| | **C1 (proposed)** | `ExecutionRegistry.ts` — `NotebookCellOutputItem.error({name, message})` instead of `.stderr(...)`, where `name = error.exception_type` when `type === "exception"` else `error.type` | Matches vscode-jupyter convention. No API contract change. Python traceback still arrives as a separate console output. | Notebook UI renders `.error()` items as a red error box rather than inline red text — slight UX change. | | C2 | `Api.ts` — add a separate completion-status yield `{success, error?}` after the loop | Explicit success/failure. | Breaks the `AsyncIterable<Output>` contract you intentionally modeled on vscode-jupyter. | | C3 | Consumers regex stderr for tracebacks | No upstream change. | Fragile; conflates warnings with failures; no `{name, message, stack}`. | **Proposed PR:** [`hugolytics:fix/error-mime-type`](https://github.com/hugolytics/marimo-lsp/tree/fix/error-mime-type) (C1). Adds a vitest case asserting `outputs[0].items[0].mime === "application/vnd.code.notebook.error"` for an exception. Updates two existing snapshots that captured the previous mime. 398/398 tests pass; 0 new dependencies. ## Broader pair-programming design — separate threads to follow These don't need decisions in this issue. Surfacing so design conversation can begin in parallel — happy to open dedicated issues if useful. ### D — agentic edit UX inside the editor What should `experimental.kernels` + companion APIs look like for operations beyond "run code": - **Cell mutation** (create / edit / delete) with **permission modes** (`auto` / `always-ask` / `bypass-permissions`) so agents can be given graduated autonomy, mirroring how Claude Code's permission system works. - **Accept/deny diff UX** for cell edits — Copilot-style inline preview, so a user can review what the agent proposes before it lands. - **Undo/redo + edit history** that interleaves agent edits and user edits cleanly. - **Git/worktree integration** — agents working on a feature branch or a separate worktree without colliding with the user's main editing flow. ### E — editor awareness for agents `experimental.kernels.getKernel(uri)` exposes the kernel but not the editor state. Agents would benefit from reading: - **Selected cells** — so they can act on "this cell" without the user repeating context. - **Cell tags** — so a user can mark cells as `do-not-edit`, `agent-scratch`, `expected-output`, etc., and agents can honor those constraints. ### A — static export integration Marimo already exports HTML/PNG/ipynb (via `marimo export`). A small companion API — or a documented pattern around the existing exporter — would let sibling extensions request an export for a given notebook URI without spawning a subprocess. Use case: agents reading rendered plots / table HTML / serialized notebook state for visual / structural inspection, without needing a headless browser in the agent. ### Bridging onboarding — where the HTTP bridge should live long-term The use case that motivates the bridge is narrowly: **a user who has both `marimo-team.vscode-marimo` AND a coding agent (Claude Code, Codex, …) and wants the agent to drive the editor's kernel.** Three other configurations don't need it at all: | Configuration | Needs bridge? | |---|---| | `marimo` standalone + agent (no VS Code) | No — marimo-pair talks to the standalone HTTP server directly | | `marimo` standalone, no agent | No | | VS Code + marimo, no agent | No | | **VS Code + marimo + agent** | **Yes** — otherwise the agent ends up driving a sidecar kernel, not the one the user sees | Today the bridge is a separate sideloaded extension (`hugolytics/agent-bridge`) — that creates onboarding friction and means the user has no signal that they should install it (silently degrades to "agent sees different state than editor"). Proposed long-term home: **bundle the bridge into `marimo-team.vscode-marimo` behind `marimo.agentBridge.enabled` (default off)**, with an onboarding nudge when the marimo extension detects a coding-agent extension installed: > *"Detected Claude Code. Want to enable marimo's agent integration? (You can flip `marimo.agentBridge.enabled` later.)"* Why not bundle in the marimo-pair skill? Because the bridge is editor-side (needs `vscode.commands.executeCommand`, `WorkspaceEdit`, the `experimental.kernels` API), while the skill is agent-side (a shell+protocol contract). Splitting them keeps each side single-responsibility. To make this concrete and friction-free for the maintainer side: [`hugolytics/agent-bridge`](https://github.com/hugolytics/agent-bridge) is ~430 LOC of TypeScript in one file, no runtime deps beyond `vscode`. The repo ships an [`INTEGRATION.md`](https://github.com/hugolytics/agent-bridge/blob/main/INTEGRATION.md) documenting a 4-step vendor: copy `src/extension.ts`, merge ~30 lines of `package.json` contributes, drop the now-redundant `activationEvents`/`extensionDependencies`, add the onboarding nudge. The current standalone extension already implements lazy activation (`onNotebook:marimo-notebook` + `extensionDependencies: ["marimo-team.vscode-marimo"]` + `agentBridge.enabled` setting) so users on the current artifact get the same UX shape as the bundled version would. **Happy to transfer the repo to `marimo-team` if that direction is preferred over a vendor.** ### Transport — HTTP vs MCP Open design question. Today the bridge speaks `marimo-pair`-compatible HTTP/SSE so the existing skill works unmodified. MCP is an alternative worth weighing: | | HTTP/SSE (today) | MCP | |---|---|---| | **Compatibility** | `marimo-pair` skill works unmodified | Requires agent to speak MCP (Claude / Codex do; arbitrary shell agents don't) | | **Streaming** | SSE; established pattern; trivially curl-able | Per-tool streaming via `tool.stream` (newer; less mature tooling) | | **Connection lifecycle** | Stateless HTTP requests; one server, many clients | Stateful connection; awkward to match VS Code extension activation/deactivation | | **State model** | URL-addressable resources (`/notebooks/cells`, `/api/sessions`) | Resource/tool/prompt model; cell-CRUD as 4 tools is fine; SSE-style streaming output of cell-ops less clean | | **Discoverability for agents** | marimo-pair-style registry file | MCP-config in agent's config | | **Maturity** | Plain HTTP, deterministic | Spec moving fast, agents implementing variably | My read: HTTP/SSE remains the right transport for now because (a) the marimo-pair skill is already a working agent-side protocol and we'd lose that compatibility on MCP, and (b) the lifecycle mismatch between MCP connections and VS Code extension activation is real. MCP becomes interesting once the editor-side API surface stabilizes — at which point both transports can coexist (bridge offers both endpoints). Worth a separate design issue if maintainers think MCP-first is the right long-term bet. --- Happy to break any of these into separate threads. --- Tagged: @manzt (from #545).