---
phase: 23-parser-proptests-and-caret-integration-tests
plan: 02
type: execute
wave: 1
depends_on: []
files_modified:
  - test/integration/test_caret_position.py
  - justfile
autonomous: true
requirements:
  - TEST-06

must_haves:
  truths:
    - "Caret renders at correct character position in DuckDB error output for a structural error (missing paren)"
    - "Caret renders at correct character position for a clause-level error (typo in keyword)"
    - "Caret renders at correct character position for a near-miss prefix error"
  artifacts:
    - path: "test/integration/test_caret_position.py"
      provides: "End-to-end caret position verification through loaded extension"
      min_lines: 60
    - path: "justfile"
      provides: "Updated test-all recipe including caret test"
      contains: "test-caret"
  key_links:
    - from: "test/integration/test_caret_position.py"
      to: "build/debug/semantic_views.duckdb_extension"
      via: "LOAD extension + execute malformed DDL"
      pattern: "LOAD.*semantic_views"
    - from: "justfile"
      to: "test/integration/test_caret_position.py"
      via: "test-caret recipe calling uv run"
      pattern: "uv run test/integration/test_caret_position.py"
---

<objective>
Create Python integration tests that verify DuckDB's caret (`^`) rendering appears at the correct character position when malformed DDL flows through the full extension load pipeline.

Purpose: Phase 21 integration tests verify error *messages* flow through the pipeline via sqllogictest, but sqllogictest `statement error` only matches message substrings -- it cannot assert on the caret line. This plan closes the gap by using Python `duckdb.ParserException` inspection to verify caret positions end-to-end.

Output: `test/integration/test_caret_position.py` with 3 representative caret position tests, plus `justfile` update to include caret tests in `test-all`.
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/23-parser-proptests-and-caret-integration-tests/23-RESEARCH.md

@test/integration/test_vtab_crash.py (established Python integration test pattern)
@test/integration/test_ducklake_helpers.py (shared helpers: get_ext_dir, get_extension_path)
@justfile (current test-all recipe)

<interfaces>
<!-- Python test helpers from test_ducklake_helpers.py -->

```python
def get_ext_dir() -> str:
    """Return the project-local extension directory."""

def get_extension_path() -> Path:
    """Return semantic_views extension path (env var or build/debug/ default)."""
```

<!-- DuckDB error format for caret rendering -->
<!-- When a ParserException is raised with error_location set, DuckDB renders: -->
<!--   Parser Error: {message}                                                -->
<!--   LINE 1: {original_query}                                                -->
<!--           ^                                                               -->
<!-- The caret line is indented to match the error position in the query.       -->
<!-- The LINE prefix is "LINE 1: " (8 chars) so caret_offset = 8 + position.  -->

<!-- Extension loading pattern (from test_vtab_crash.py): -->
```python
EXT_DIR = get_ext_dir()
EXT_PATH = get_extension_path()

def make_connection():
    conn = duckdb.connect(config={
        "allow_unsigned_extensions": "true",
        "extension_directory": EXT_DIR,
    })
    conn.execute(f"FORCE INSTALL '{EXT_PATH}'")
    conn.execute("LOAD semantic_views")
    return conn
```
</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Create test_caret_position.py with 3 representative caret tests</name>
  <files>test/integration/test_caret_position.py</files>
  <action>
Create `test/integration/test_caret_position.py` following the `test_vtab_crash.py` pattern (PEP 723 inline dependencies, uv-runnable, sys.exit-based).

Structure:

1. **PEP 723 script header:**
   ```python
   # /// script
   # dependencies = ["duckdb"]
   # requires-python = ">=3.9"
   # ///
   ```

2. **Imports and helpers:** Import from `test_ducklake_helpers` (same pattern as `test_vtab_crash.py`). Create `make_connection()` helper that loads the extension.

3. **Caret extraction helper:**
   ```python
   def extract_caret_position(error_text: str) -> int | None:
       """Extract the caret (^) column position from DuckDB error output.

       DuckDB renders errors as:
           Parser Error: {message}
           LINE 1: {query_text}
                   ^

       Returns the 0-based column position of ^ relative to the query text
       (subtracting the "LINE 1: " prefix of 8 characters), or None if no
       caret line found.
       """
   ```
   Parse the error string line by line. Find the line containing only whitespace and `^`. Compute the column position by finding the index of `^` and subtracting the "LINE 1: " prefix length (8 chars). Return 0-based offset into the query text.

4. **Test 1: Structural error -- missing opening paren (ERR-02 representative):**
   ```python
   def test_caret_missing_paren():
       """Caret points where '(' is expected after view name."""
       conn = make_connection()
       try:
           conn.execute("CREATE SEMANTIC VIEW myview tables := []")
       except duckdb.ParserException as e:
           error_text = str(e)
           assert "Expected '('" in error_text
           pos = extract_caret_position(error_text)
           assert pos is not None, f"No caret found in: {error_text}"
           # The caret should point at 'tables' (position after "CREATE SEMANTIC VIEW myview ")
           # "CREATE SEMANTIC VIEW myview " = 28 chars, so position should be around 28
           query = "CREATE SEMANTIC VIEW myview tables := []"
           assert query[pos:pos+6] == "tables", f"Caret at {pos}, expected 'tables' but got '{query[pos:pos+6]}'"
   ```

5. **Test 2: Clause-level error -- typo in keyword (ERR-01 representative):**
   ```python
   def test_caret_clause_typo():
       """Caret points at misspelled clause keyword."""
       conn = make_connection()
       try:
           conn.execute("CREATE SEMANTIC VIEW x (tbles := [], dimensions := [])")
       except duckdb.ParserException as e:
           error_text = str(e)
           # Should suggest "tables"
           assert "tables" in error_text.lower()
           pos = extract_caret_position(error_text)
           assert pos is not None
           query = "CREATE SEMANTIC VIEW x (tbles := [], dimensions := [])"
           assert query[pos:pos+5] == "tbles", f"Caret at {pos}, expected 'tbles' but got '{query[pos:pos+5]}'"
   ```

6. **Test 3: Near-miss prefix error (ERR-03 representative):**
   ```python
   def test_caret_near_miss():
       """Caret points at start of near-miss prefix."""
       conn = make_connection()
       try:
           conn.execute("CRETAE SEMANTIC VIEW x (tables := [])")
       except duckdb.ParserException as e:
           error_text = str(e)
           # Should suggest "CREATE SEMANTIC VIEW"
           assert "CREATE SEMANTIC VIEW" in error_text
           pos = extract_caret_position(error_text)
           assert pos is not None
           # Caret should point at position 0 (start of statement)
           assert pos == 0, f"Caret at {pos}, expected 0"
   ```

7. **Main runner:** Same pattern as `test_vtab_crash.py` -- iterate tests, print results, sys.exit(0) on all pass, sys.exit(1) on any failure.

8. **Important:** The caret line format may vary slightly. The `extract_caret_position` function should be robust: find any line that contains `^` and is NOT the `LINE 1:` line or the `Parser Error:` line. If DuckDB doesn't render the caret for a particular error (e.g., near-miss might go through a different code path), document the behavior and adjust the test to match what actually happens.

9. **Build requirement:** The test requires `just build` first since it loads the extension binary. The justfile recipe will handle this.
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && just build && uv run test/integration/test_caret_position.py 2>&1 | tail -20</automated>
  </verify>
  <done>3 Python caret position tests pass. Each verifies the caret renders at the correct character position for a different error type (structural, clause-level, near-miss).</done>
</task>

<task type="auto">
  <name>Task 2: Add test-caret recipe to justfile and include in test-all</name>
  <files>justfile</files>
  <action>
Update the `justfile` to add a `test-caret` recipe and include it in `test-all`:

1. **Add `test-caret` recipe** (after the `test-vtab-crash` recipe):
   ```
   # Run Python caret position integration tests against the built extension.
   # Verifies that DuckDB error caret (^) renders at the correct character position
   # when malformed DDL flows through the extension's parser hook pipeline.
   test-caret: build
       uv run test/integration/test_caret_position.py
   ```

2. **Update `test-all` recipe** to include `test-caret`:
   Change from:
   ```
   test-all: test-rust test-sql test-ducklake-ci test-vtab-crash
   ```
   To:
   ```
   test-all: test-rust test-sql test-ducklake-ci test-vtab-crash test-caret
   ```

3. **Update the comment** on `test-all` to mention caret tests:
   ```
   # Run all tests: Rust unit tests + SQL logic tests + DuckLake integration + vtab crash + caret position
   ```
  </action>
  <verify>
    <automated>cd /Users/paul/Documents/Dev/Personal/duckdb-semantic-views && just test-all 2>&1 | tail -30</automated>
  </verify>
  <done>`just test-all` includes and passes the caret position tests. The `test-caret` recipe is available as a standalone command.</done>
</task>

</tasks>

<verification>
Run the caret position tests standalone:
```bash
just test-caret
```

Run the full test suite to confirm integration:
```bash
just test-all
```
</verification>

<success_criteria>
- `test/integration/test_caret_position.py` exists with 3 caret position tests
- `just test-caret` passes (all 3 tests green)
- `just test-all` passes (includes caret tests, no regressions)
- Each test verifies the caret character position, not just the error message text
</success_criteria>

<output>
After completion, create `.planning/phases/23-parser-proptests-and-caret-integration-tests/23-02-SUMMARY.md`
</output>
