---
phase: 11-create-semantic-view-parser-hook
plan: 02
type: execute
wave: 2
depends_on: [11-01]
files_modified:
  - src/shim/shim.h
  - src/shim/shim.cpp
autonomous: true
requirements: [DDL-01, DDL-02, DDL-03, DDL-06]

must_haves:
  truths:
    - "CREATE SEMANTIC VIEW name TABLES (...) DIMENSIONS (...) METRICS (...) compiles without error and is parseable by the parse_function_t"
    - "DROP SEMANTIC VIEW name and DROP SEMANTIC VIEW IF EXISTS name are handled by the parse_function_t"
    - "SELECT 1 and all native DuckDB SQL bypasses the parser hook without error (DDL-06)"
    - "RELATIONSHIPS clause parses from_alias(fk_cols) REFERENCES ref_alias correctly"
    - "Nested function calls in DIMENSIONS/METRICS/FACTS (e.g., COALESCE(a, b)) parse without premature comma-split"
    - "CREATE OR REPLACE SEMANTIC VIEW calls catalog_upsert FFI; plain CREATE calls catalog_insert FFI"
  artifacts:
    - path: "src/shim/shim.h"
      provides: "Declarations for four semantic_views_catalog_* FFI functions; semantic_views_register_shim signature unchanged"
    - path: "src/shim/shim.cpp"
      provides: "Full parser hook: SemanticViewsParserInfo, SemanticViewsDDLData, SemanticViewsParseFunction, SemanticViewsPlanFunction, SemanticViewsDDLBind, SemanticViewsDDLScan, ParseSemanticViewStatement tokenizer, registration in semantic_views_register_shim"
  key_links:
    - from: "src/shim/shim.cpp SemanticViewsDDLScan"
      to: "src/catalog.rs ffi_catalog functions"
      via: "semantic_views_catalog_insert / _upsert / _delete / _delete_if_exists FFI calls"
      pattern: "semantic_views_catalog_"
    - from: "src/shim/shim.cpp SemanticViewsDDLScan"
      to: "src/shim/shim.cpp semantic_views_pragma_define/_drop"
      via: "persist_conn write via persist connection before in-memory catalog update"
      pattern: "semantic_views_pragma_define|semantic_views_pragma_drop"
    - from: "src/shim/shim.cpp semantic_views_register_shim"
      to: "DBConfig::GetConfig(db_instance).parser_extensions"
      via: "config.parser_extensions.push_back(ext)"
      pattern: "parser_extensions\\.push_back"
    - from: "src/shim/shim.cpp SemanticViewsPlanFunction"
      to: "src/shim/shim.cpp SemanticViewsDDLBind + SemanticViewsDDLScan"
      via: "if_not_exists passed as result.parameters[4] (Value(bool)), extracted in bind, checked in scan after catalog_insert returns -1 to silently return rather than throw"
      pattern: "if_not_exists"
---

<objective>
Implement the C++ parser extension hook that handles CREATE SEMANTIC VIEW and DROP SEMANTIC VIEW DDL statements.

Purpose: The parser hook is the core of Phase 11 — it converts native SQL DDL into internal catalog operations without modifying DuckDB core code.

Output: Updated shim.h with catalog FFI declarations; updated shim.cpp with full parse_function_t + plan_function_t + DDL execution via TableFunction scan.
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/phases/11-create-semantic-view-parser-hook/11-CONTEXT.md
@.planning/phases/11-create-semantic-view-parser-hook/11-RESEARCH.md
@.planning/phases/11-create-semantic-view-parser-hook/11-01-SUMMARY.md

<interfaces>
<!-- Key contracts for the C++ implementation. -->

From src/shim/shim.h (current):
```c
void semantic_views_register_shim(void* db_instance_ptr);
int32_t semantic_views_pragma_define(duckdb_connection conn, const char* name, const char* json);
int32_t semantic_views_pragma_drop(duckdb_connection conn, const char* name);
```

From src/shim/shim.cpp (current):
```cpp
// semantic_views_register_shim: casts db_instance_ptr → DatabaseInstance&
//   → ExtensionLoader → registers two PRAGMA callbacks
// semantic_views_pragma_define: uses persist conn to INSERT OR REPLACE
// semantic_views_pragma_drop: uses persist conn to DELETE
// Pattern: duckdb_query(conn, sql, &result) for direct SQL via C API
```

From src/catalog.rs (after plan 11-01):
```rust
// FFI functions to declare in shim.h:
pub unsafe extern "C" fn semantic_views_catalog_insert(
    catalog_ptr: *const CatalogState, name_ptr: *const c_char, json_ptr: *const c_char
) -> i32;
pub unsafe extern "C" fn semantic_views_catalog_upsert(...) -> i32;
pub unsafe extern "C" fn semantic_views_catalog_delete(...) -> i32;
pub unsafe extern "C" fn semantic_views_catalog_delete_if_exists(...) -> i32;
// CatalogState = Arc<RwLock<HashMap<String, String>>> (opaque to C++)
```

From 11-RESEARCH.md (verified API):
```cpp
// ParserExtension registration:
ParserExtension ext;
ext.parse_function = SemanticViewsParseFunction;
ext.plan_function = SemanticViewsPlanFunction;
ext.parser_info = make_shared_ptr<SemanticViewsParserInfo>(...);
config.parser_extensions.push_back(ext);

// parse_function_t signature:
// ParserExtensionParseResult (*)(ParserExtensionInfo*, const string&)
// Return ParserExtensionParseResult() for fall-through (DISPLAY_ORIGINAL_ERROR)
// Return ParserExtensionParseResult("msg") for parse errors
// Return ParserExtensionParseResult(data_ptr) for success

// plan_function_t signature:
// ParserExtensionPlanResult (*)(ParserExtensionInfo*, ClientContext&,
//                               unique_ptr<ParserExtensionParseData>)
// CRITICAL: DO NOT call context.Query() from plan function or scan — DEADLOCK
// Use persist_conn for all SQL writes from the scan function

// StatementReturnType::NOTHING = DDL returns no rows
// result.modified_databases["main"] = {} to signal write operation
```

Base table determination (Claude's Discretion resolution from RESEARCH.md):
- Base table = the table alias NOT referenced as the target of any REFERENCES clause
- If no RELATIONSHIPS clause, first declared TABLES entry is base table
- If all tables referenced (circular), use first declared table
</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Update shim.h — add catalog FFI declarations</name>
  <files>src/shim/shim.h</files>
  <action>
  Add the four catalog FFI function declarations to `src/shim/shim.h`, after the existing `semantic_views_pragma_drop` declaration. These are defined in Rust (catalog.rs) and called from C++ scan function:

  ```c
  // ---------------------------------------------------------------------------
  // Catalog mutation FFI — implemented in Rust (src/catalog.rs ffi_catalog).
  // Called from the C++ parser hook scan function to update the in-memory catalog.
  // catalog_ptr is an opaque pointer to the Rust CatalogState (Arc<RwLock<HashMap>>).
  // All functions return 0 on success, -1 on error.
  // ---------------------------------------------------------------------------

  /// Insert a new semantic view. Returns -1 if view already exists or json is invalid.
  int32_t semantic_views_catalog_insert(
      const void* catalog_ptr,
      const char* name,
      const char* json
  );

  /// Upsert a semantic view (insert or replace). Returns -1 if json is invalid.
  int32_t semantic_views_catalog_upsert(
      const void* catalog_ptr,
      const char* name,
      const char* json
  );

  /// Delete a semantic view. Returns -1 if view does not exist.
  int32_t semantic_views_catalog_delete(
      const void* catalog_ptr,
      const char* name
  );

  /// Delete a semantic view if it exists. Silently succeeds if absent. Returns 0 always.
  int32_t semantic_views_catalog_delete_if_exists(
      const void* catalog_ptr,
      const char* name
  );
  ```

  Also update the comment on `semantic_views_register_shim` to reflect that Phase 11 adds parser hooks (this comment already says "Phase 11 will add" — change to "Phase 11: adds").
  </action>
  <verify>
    <automated>grep -n "semantic_views_catalog_" src/shim/shim.h | wc -l</automated>
  </verify>
  <done>shim.h contains all four semantic_views_catalog_* declarations inside extern "C" block</done>
</task>

<task type="auto">
  <name>Task 2: Implement C++ parser hook in shim.cpp</name>
  <files>src/shim/shim.cpp</files>
  <action>
  Add the full parser extension implementation to `src/shim/shim.cpp`. Add all new code BEFORE the existing `extern "C" {` block (the C-linkage functions), inserting it after the PRAGMA callbacks section.

  **Step 1: Update file header comment** to reflect Phase 11 additions.

  **Step 2: Add `semantic_views_register_shim` extension parameter** — update the function signature to accept two additional parameters for the catalog raw pointer and persist_conn. Since lib.rs (plan 11-03) will update the Rust call site, coordinate: change signature to:
  ```c
  void semantic_views_register_shim(
      void* db_instance_ptr,
      const void* catalog_raw_ptr,    // raw ptr to Rust CatalogState (opaque to C++)
      duckdb_connection persist_conn_param  // pre-created persist connection
  );
  ```
  Update shim.h to match this new signature.

  **Step 3: Add the C++ data structures and parser hook** (add this as a new `namespace` block or as static file-scope items before `extern "C" {`):

  ```cpp
  // ---------------------------------------------------------------------------
  // Phase 11: CREATE SEMANTIC VIEW / DROP SEMANTIC VIEW parser hook.
  // Implements DuckDB's ParserExtension mechanism.
  //
  // Critical constraints (verified from DuckDB source):
  //   - parse_function_t is called for every statement that native parser fails.
  //     MUST return ParserExtensionParseResult() (DISPLAY_ORIGINAL_ERROR) for
  //     all non-matching input — never DISPLAY_EXTENSION_ERROR for non-CSV DDL.
  //   - plan_function_t is called with context_lock held — NEVER call
  //     context.Query() from plan function OR scan function (deadlock).
  //   - Use persist_conn for all SQL writes from the scan function.
  // ---------------------------------------------------------------------------

  // DDL operation type
  enum class SemanticViewsDDLType : int32_t { CREATE = 0, DROP = 1 };

  // Parse data — carries fully parsed DDL statement from parse_function_t to plan_function_t
  struct SemanticViewsDDLData : ParserExtensionParseData {
      SemanticViewsDDLType ddl_type;
      bool or_replace = false;
      bool if_not_exists = false;
      bool if_exists = false;
      std::string view_name;
      std::string definition_json; // serialized JSON for CREATE; empty for DROP

      unique_ptr<ParserExtensionParseData> Copy() const override {
          return make_unique<SemanticViewsDDLData>(*this);
      }
      std::string ToString() const override {
          return "SemanticViewsDDL:" + view_name;
      }
  };

  // Parser extension info — carries catalog pointer and persist_conn for scan access
  struct SemanticViewsParserInfo : ParserExtensionInfo {
      const void* catalog_ptr;          // Rust CatalogState raw pointer (opaque)
      duckdb_connection persist_conn;   // pre-created separate connection for SQL writes
      SemanticViewsParserInfo(const void* cat, duckdb_connection pc)
          : catalog_ptr(cat), persist_conn(pc) {}
  };

  // Bind data for the TableFunction scan
  struct SemanticViewsDDLBindData : FunctionData {
      SemanticViewsDDLType ddl_type;
      bool or_replace;
      bool if_not_exists;   // CREATE ... IF NOT EXISTS — silently succeed if view already exists
      bool if_exists;
      std::string view_name;
      std::string definition_json;
      const void* catalog_ptr;
      duckdb_connection persist_conn;

      unique_ptr<FunctionData> Copy() const override {
          return make_unique<SemanticViewsDDLBindData>(*this);
      }
      bool Equals(const FunctionData &other) const override { return true; }
  };
  ```

  **Step 4: Implement the hand-written DDL tokenizer** `ParseSemanticViewStatement`:

  The tokenizer must handle the full Snowflake-compatible grammar. Key rules:
  - Uppercase the input once for keyword matching; preserve original case for expressions and names
  - Track parenthesis depth when consuming expression lists — only split on comma at depth 0
  - Each clause item: `alias.name AS sql_expr` for DIMENSIONS/METRICS/FACTS, `alias AS table [PRIMARY KEY (cols)]` for TABLES, `from_alias(fk_cols) REFERENCES ref_alias` for RELATIONSHIPS
  - Base table = table alias NOT appearing as a `from_alias` in any REFERENCES clause; if no RELATIONSHIPS, first TABLES entry is base table
  - Build a JSON string for the definition using string concatenation (no JSON library needed — values are already validated by the subsequent Rust parse)
  - Parse-time validation: verify that all aliases used in DIMENSIONS/METRICS/FACTS/RELATIONSHIPS are declared in the TABLES clause; throw `std::runtime_error` on violation

  **Step 5: Implement `SemanticViewsParseFunction`** (matches `parse_function_t`):
  ```cpp
  static ParserExtensionParseResult SemanticViewsParseFunction(
      ParserExtensionInfo *info, const std::string &query) {
      // DDL-06: fast keyword check — return DISPLAY_ORIGINAL_ERROR for non-matching
      std::string upper = StringUtil::Upper(query);
      bool is_create = (upper.find("CREATE") != std::string::npos &&
                        upper.find("SEMANTIC") != std::string::npos);
      bool is_drop   = (upper.find("DROP") != std::string::npos &&
                        upper.find("SEMANTIC") != std::string::npos);
      if (!is_create && !is_drop) {
          return ParserExtensionParseResult(); // DISPLAY_ORIGINAL_ERROR — fall through
      }
      try {
          auto data = ParseSemanticViewStatement(query);
          return ParserExtensionParseResult(std::move(data));
      } catch (const std::exception &e) {
          return ParserExtensionParseResult(std::string(e.what()));
      }
  }
  ```

  **Step 6: Implement `SemanticViewsPlanFunction`** (matches `plan_function_t`):
  - Cast `info_base` to `SemanticViewsParserInfo`
  - Build a `TableFunction` with name `"semantic_view_ddl_exec"`, function pointer `SemanticViewsDDLScan`, bind pointer `SemanticViewsDDLBind`
  - Pass DDL data as `result.parameters` in this order (bind extracts by index):
    - `Value(ddl.view_name)` — index 0
    - `Value(ddl.definition_json)` — index 1
    - `Value((int32_t)ddl.ddl_type)` — index 2
    - `Value(ddl.or_replace)` — index 3
    - `Value(ddl.if_not_exists)` — index 4 (**required**: bind/scan use this for IF NOT EXISTS silent-succeed path)
    - `Value(ddl.if_exists)` — index 5
    - `Value::POINTER((uintptr_t)info.catalog_ptr)` — index 6
    - `Value::POINTER((uintptr_t)info.persist_conn)` — index 7
  - Set `result.return_type = StatementReturnType::NOTHING`
  - Set `result.modified_databases["main"] = {}`
  - Set `result.requires_valid_transaction = true`

  **Step 7: Implement `SemanticViewsDDLBind`** and **`SemanticViewsDDLScan`**:

  `SemanticViewsDDLBind`: extracts parameters into `SemanticViewsDDLBindData` by index (matching the order in Step 6):
  - `bind_data.view_name = input.inputs[0].GetValue<string>()`
  - `bind_data.definition_json = input.inputs[1].GetValue<string>()`
  - `bind_data.ddl_type = (SemanticViewsDDLType)input.inputs[2].GetValue<int32_t>()`
  - `bind_data.or_replace = input.inputs[3].GetValue<bool>()`
  - `bind_data.if_not_exists = input.inputs[4].GetValue<bool>()`
  - `bind_data.if_exists = input.inputs[5].GetValue<bool>()`
  - `bind_data.catalog_ptr = (const void*)input.inputs[6].GetPointer()`
  - `bind_data.persist_conn = (duckdb_connection)input.inputs[7].GetPointer()`
  Returns `make_unique<SemanticViewsDDLBindData>(...)`.

  `SemanticViewsDDLScan`: executes the DDL operation:
  ```cpp
  static void SemanticViewsDDLScan(ClientContext &context,
                                    TableFunctionInput &data, DataChunk &output) {
      auto &bind_data = data.bind_data->Cast<SemanticViewsDDLBindData>();
      output.SetCardinality(0);

      if (bind_data.ddl_type == SemanticViewsDDLType::CREATE) {
          // 1. Persist: write to semantic_layer._definitions via persist_conn
          int rc = semantic_views_pragma_define(bind_data.persist_conn,
                                                bind_data.view_name.c_str(),
                                                bind_data.definition_json.c_str());
          if (rc != 0 && !bind_data.or_replace) {
              throw std::runtime_error("Failed to persist semantic view '" +
                                       bind_data.view_name + "'");
          }
          // 2. Update in-memory catalog via Rust FFI
          int cat_rc;
          if (bind_data.or_replace) {
              cat_rc = semantic_views_catalog_upsert(bind_data.catalog_ptr,
                  bind_data.view_name.c_str(),
                  bind_data.definition_json.c_str());
          } else {
              cat_rc = semantic_views_catalog_insert(bind_data.catalog_ptr,
                  bind_data.view_name.c_str(),
                  bind_data.definition_json.c_str());
          }
          if (cat_rc != 0) {
              // IF NOT EXISTS: silently succeed when view already exists (cat_rc == -1)
              if (bind_data.if_not_exists) {
                  output.SetCardinality(0);
                  return;
              }
              throw std::runtime_error("Semantic view '" + bind_data.view_name +
                                       "' already exists");
          }
      } else { // DROP
          // 1. Persist: remove from semantic_layer._definitions
          semantic_views_pragma_drop(bind_data.persist_conn,
                                     bind_data.view_name.c_str());
          // 2. Update in-memory catalog
          if (bind_data.if_exists) {
              semantic_views_catalog_delete_if_exists(bind_data.catalog_ptr,
                  bind_data.view_name.c_str());
          } else {
              int cat_rc = semantic_views_catalog_delete(bind_data.catalog_ptr,
                  bind_data.view_name.c_str());
              if (cat_rc != 0) {
                  throw std::runtime_error("Semantic view '" + bind_data.view_name +
                                           "' does not exist");
              }
          }
      }
  }
  ```

  **Step 8: Register parser hook in `semantic_views_register_shim`** — after existing PRAGMA registration, add:
  ```cpp
  // Register parser extension for CREATE/DROP SEMANTIC VIEW (Phase 11)
  auto &config = DBConfig::GetConfig(db_instance);
  ParserExtension parser_ext;
  parser_ext.parse_function = SemanticViewsParseFunction;
  parser_ext.plan_function = SemanticViewsPlanFunction;
  parser_ext.parser_info = make_shared_ptr<SemanticViewsParserInfo>(
      catalog_raw_ptr, persist_conn_param);
  config.parser_extensions.push_back(parser_ext);
  ```

  Note: The `TableFunctionBindInput` / bind function pattern for getting catalog_ptr and persist_conn into the scan function: store them in `SemanticViewsDDLBindData` during the bind step. The bind function receives the `ParserExtensionPlanResult.parameters` vector as `input.inputs`. Additionally store the `SemanticViewsParserInfo*` pointer in the bind data by passing it via an extra parameter or by casting `input.input_table_function.function_info` (if that pointer is available in bind context).

  Simplest approach (confirmed in RESEARCH.md): The `SemanticViewsPlanFunction` passes catalog_ptr and persist_conn as additional `Value` parameters alongside the DDL data parameters. The bind function extracts them via `input.inputs[N]` as `GetPointer()` / `GetValue<int64_t>()` (cast to pointer). Use `Value::POINTER(uintptr_t)` to pass raw pointers through DuckDB's Value system safely:
  - `result.parameters.push_back(Value::POINTER((uintptr_t)info.catalog_ptr))`
  - `result.parameters.push_back(Value::POINTER((uintptr_t)info.persist_conn))`

  Then in bind: `bind_data.catalog_ptr = (const void*)input.inputs[N].GetPointer()`.

  Verify the build compiles before ending the task:
  ```bash
  cargo build --features extension 2>&1 | tail -20
  ```
  </action>
  <verify>
    <automated>cargo build --features extension 2>&1 | grep -E "^error|finished|warning\[" | tail -20</automated>
  </verify>
  <done>cargo build --features extension completes without errors; shim.cpp contains SemanticViewsParseFunction, SemanticViewsPlanFunction, SemanticViewsDDLScan, ParseSemanticViewStatement tokenizer, and parser hook registration in semantic_views_register_shim</done>
</task>

</tasks>

<verification>
Build check:
```bash
cargo build --features extension 2>&1 | grep "^error" | wc -l
```
Expected: 0 errors.

Symbol check:
```bash
nm -gU target/debug/libsemantic_views.dylib 2>/dev/null | grep -E "semantic_views_catalog_|SemanticViews" | head -10
```
Expected: catalog FFI symbols and C++ mangled symbols visible.
</verification>

<success_criteria>
- `cargo build --features extension` succeeds with zero errors
- `src/shim/shim.h` declares all four `semantic_views_catalog_*` functions
- `src/shim/shim.cpp` has the full parser hook: parse_function_t, plan_function_t, bind, scan, tokenizer, registration
- `semantic_views_register_shim` signature updated to accept `catalog_raw_ptr` and `persist_conn_param`
- DDL-06: parse_function_t returns DISPLAY_ORIGINAL_ERROR (not DISPLAY_EXTENSION_ERROR) for non-matching SQL
</success_criteria>

<output>
After completion, create `.planning/phases/11-create-semantic-view-parser-hook/11-02-SUMMARY.md`
</output>
