---
phase: 58-transactional-ddl
plan: 01
type: retroactive-reconstruction
wave: 1
depends_on: []
autonomous: false
requirements: [TXN-DDL-01, TXN-DDL-02, TXN-DDL-03, TXN-DDL-04]
source_commits:
  - 680a967  # feat(parse): add parser_override hook for transactional CREATE DDL (v0.8.0 foundation)
  - 7c9b4df  # refactor(catalog): drop CatalogState HashMap; reads go through CatalogReader
  - 55ddcda  # fix(cpp): include trailing ParserOptions field in Parser compat shim
  - f9f22b3  # feat(parse): native DROP/ALTER rewrites via parser_override
  - f432f1e  # feat(parse): transactional CREATE via parser_override
  - 8ef23d0  # feat(parse): transactional CREATE FROM YAML FILE via parser_override
  - 5b1d243  # test: add v080_yaml_file.yaml fixture
  - 6a25b40  # chore: close v0.8.0 — ADBC test, example, CHANGELOG, version bump
  - 225b71c  # fix(ffi): heap-owned SQL buffers — no silent truncation
  - 28b3291  # fix(v0.8.0): address PR #28 review — multi-DB isolation, friendly dup error, PEG YAML FILE coverage
files_modified:
  - cpp/include/parser_extension_compat.hpp
  - cpp/src/shim.cpp
  - src/parse.rs
  - src/catalog.rs
  - src/lib.rs
  - test/sql/v080_transactional_ddl.test
  - test/integration/test_adbc_transactions.py
  - examples/transactional_ddl.py
---

# Phase 58: Transactional DDL via `parser_override`

> Reconstructed retroactively on 2026-05-05 from the ad-hoc v0.8.0 work that was never registered as a GSD phase. Source commits, file lists, and verification cite the real artefacts that landed on `milestone/v0.8.0` (PR #28). No RESEARCH.md or DISCUSS.md exists for this phase — those records were not produced during the original ad-hoc work.

## Goal

`CREATE` (all four forms), `DROP`, and `ALTER SEMANTIC VIEW` participate in the caller's transaction. `BEGIN ... ROLLBACK` rolls back uncommitted catalog changes; `BEGIN ... COMMIT` persists them. ADBC, dbt, and other transaction-aware clients see the contract they expect.

## Approach

Register a DuckDB `parser_override` extension hook (introduced for this purpose). The hook fires before the default parser; for recognised DDL it produces a rewritten native SQL statement (`INSERT` / `UPDATE` / `DELETE` against `semantic_layer._definitions`) which DuckDB then re-parses and executes on the **caller's connection** — so the DML is naturally part of the caller's transaction. Non-matching statements return a default-constructed `ParserOverrideResult` (= `DISPLAY_ORIGINAL_ERROR`), letting the default parser proceed unchanged.

Architectural changes that fall out of the rewrite:

- **`CatalogState` HashMap removed** (commit 7c9b4df). All catalog reads go through a single shared `CatalogReader` that queries `_definitions` directly. The old write-through-both pattern carried a divergence risk between the in-memory mirror and the on-disk table; the rewrite path makes the mirror redundant.
- **All four `CREATE` forms wired through one rewrite path** (f432f1e, 8ef23d0): inline `AS` keyword body, inline `FROM YAML $$ ... $$`, `FROM YAML FILE '<path>'` (httpfs / S3 supported), and `CREATE OR REPLACE` / `CREATE IF NOT EXISTS`.
- **Heap-owned FFI SQL buffers** (225b71c) — the C++ → Rust trampoline previously truncated rewritten SQL at 64 KB on the stack; now the buffer is heap-allocated and freed by C++ after parsing.
- **Multi-DB isolation via `db_token`** (28b3291): `SemanticViewsParserInfo` carries a per-load token so two DuckDB instances in the same process don't cross-talk through a global catalog map.

## Files modified

- `cpp/include/parser_extension_compat.hpp` — `ParserOverrideResult`, `parser_override` typedef, `ParserOptions` size.
- `cpp/src/shim.cpp` — `sv_parser_override` C++ wrapper, registration via `SetOption("allow_parser_override_extension", "FALLBACK")`.
- `src/parse.rs` — `sv_parser_override_rust` FFI entry, `validate_and_rewrite`, `rewrite_create` / `rewrite_drop_or_alter` / `rewrite_yaml_file_create`.
- `src/catalog.rs` — `CatalogReader` query helpers replacing the HashMap mirror.
- `src/lib.rs` — `init_extension` registers the parser hook with a per-load `catalog_conn`.

## Tests

- `test/sql/v080_transactional_ddl.test` — `BEGIN; CREATE; ROLLBACK; SELECT NOT EXISTS` etc., for inline + FROM YAML + DROP + ALTER.
- `test/integration/test_adbc_transactions.py` — exercises `autocommit=False` rollback / commit semantics through the ADBC driver, end-to-end.
- `examples/transactional_ddl.py` — narrated end-to-end example.

## Known follow-ups (handled in later phases)

- The pre-rewrite catalog snapshot read uses a separate connection, so DROP/ALTER under concurrent commit would silently no-op → addressed in Phase 60 (race guards).
- The multi-DB token map is unbounded → bounded LRU added in Phase 61.
- Validation errors flow through `DISPLAY_EXTENSION_ERROR`, which `FALLBACK_OVERRIDE` mode silently drops → exposed by Phase 59 (unification removed the fallback `parse_function`); workaround in Phase 60; structural fix slated for Phase 62.
