---
phase: 13-type-mapping-and-property-based-tests-for-typed-column-dispatch
plan: 02
type: tdd
wave: 2
depends_on:
  - 13-01
files_modified:
  - tests/output_proptest.rs
autonomous: true
requirements: []

must_haves:
  truths:
    - "TIMESTAMP columns verified to return non-NULL values via integration PBT"
    - "BOOLEAN columns verified to read/write correct 0/1 values via unit PBT"
    - "DECIMAL columns verified to return correct DECIMAL typed output via integration PBT"
    - "LIST(BIGINT) columns verified to return LIST typed output (not VARCHAR) via integration PBT"
    - "NULL source rows produce NULL output rows for all tested types"
    - "All integer types (TINYINT through UBIGINT) roundtrip correctly via unit PBT"
    - "FLOAT and DOUBLE roundtrip correctly via unit PBT"
    - "DATE binary read returns correct days-since-epoch value via unit PBT"
    - "Integration PBT test suite completes in under 30 seconds (cases: 20)"
  artifacts:
    - path: "tests/output_proptest.rs"
      provides: "Unit PBTs for binary-read helpers and integration PBTs for full roundtrip"
      min_lines: 150
  key_links:
    - from: "tests/output_proptest.rs"
      to: "src/query/table_function.rs"
      via: "semantic_views::query::table_function re-exports or inline Connection roundtrip"
      pattern: "Connection::open_in_memory"
---

<objective>
Create `tests/output_proptest.rs` with two layers of property-based tests:
1. **Unit PBTs** — validate binary-read helper behaviours for all scalar types using in-memory DuckDB result chunks (SELECT literal value, execute_sql_raw, read_typed_from_vector)
2. **Integration PBTs** — full roundtrip via in-memory DuckDB: generate values → INSERT → create_semantic_view → semantic_view() → assert output type and values match

Purpose: The refactor in Plan 01 eliminates the VARCHAR cast wrapper but introduces new binary read code paths that are subtle and type-specific. Tests ensure correctness for boundary values, NULLs, and each type category. Without these tests, silent regressions (the same class of bug as TIMESTAMP all-NULL) would only surface in production.

Output: `tests/output_proptest.rs` containing all Phase 13 PBTs. All tests pass. `cargo test` completes in under 60 seconds.
</objective>

<execution_context>
@/Users/paul/.claude/get-shit-done/workflows/execute-plan.md
@/Users/paul/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/13-type-mapping-and-property-based-tests-for-typed-column-dispatch/13-CONTEXT.md
@.planning/phases/13-type-mapping-and-property-based-tests-for-typed-column-dispatch/13-RESEARCH.md
@.planning/phases/13-type-mapping-and-property-based-tests-for-typed-column-dispatch/13-01-SUMMARY.md

<interfaces>
<!-- Key patterns from existing test infrastructure -->

From tests/expand_proptest.rs:
```rust
use proptest::prelude::*;
use semantic_views::expand::{expand, QueryRequest};
use semantic_views::model::{Dimension, Join, Metric, SemanticViewDefinition};

proptest! {
    #[test]
    fn my_property(values in prop::collection::vec(any::<i64>(), 1..10)) {
        // property assertion using prop_assert!
    }
}
```

For integration PBTs with DuckDB roundtrip:
```rust
use duckdb::Connection;

// Open in-memory DB (works with default features = bundled):
let conn = Connection::open_in_memory().unwrap();

// Load the extension:
conn.execute_batch("LOAD 'target/debug/libsemantic_views.dylib'").unwrap();
// OR: register the VTab directly using the internal extension_entrypoint hook
// (this avoids needing the .dylib file — preferred for CI portability)
```

Key constraint from RESEARCH.md:
The VTab (SemanticViewVTab) can only be registered via `extension_entrypoint`. Integration tests
must register the VTab using lib.rs's extension_entrypoint function or via
Connection::register_table_function(). The bundled feature enables Connection::open_in_memory().

Pattern for in-process registration (no .dylib needed):
```rust
// In tests/output_proptest.rs — use the #[cfg(not(feature = "extension"))] guard
// semantic_views::extension_entrypoint_bundled or expose a test helper
// OR: use Connection::open_in_memory() + raw execute_sql_raw to bypass VTab entirely
// and test the binary read helpers directly by calling through the C API
```

Simplest integration test approach (avoids VTab registration complexity):
1. Use Connection::open_in_memory() to create a table and insert values
2. Access the raw connection handle via unsafe transmute or lib.rs helper
3. Call execute_sql_raw() directly on expanded SQL (bypassing the VTab)
4. Call read_typed_from_vector() on the result chunks
5. Assert the TypedValues match expected values

If semantic_views::query::table_function::read_typed_from_vector is not pub(crate),
make it pub(crate) to enable integration test access.

For NULL tests: insert NULL values via SQL `INSERT INTO t VALUES (NULL)` and assert
TypedValue::Null is returned.

For the TIMESTAMP bug regression test:
```rust
// This should return TypedValue::I64(microseconds) not TypedValue::Null
let conn = Connection::open_in_memory().unwrap();
// Get raw connection handle
// execute_sql_raw: SELECT TIMESTAMP '2024-01-15 10:30:00'
// read_typed_from_vector: must return TypedValue::I64(x) where x != 0
```

For the BOOLEAN UB regression test:
```rust
// TypedValue::Bool(true/false) — no UB
// write_typed_column with BOOLEAN type_id must write u8 into BOOLEAN slot
```

Proptest config for integration tests (keep suite fast):
```rust
proptest! {
    #![proptest_config(ProptestConfig { cases: 20, .. ProptestConfig::default() })]
    #[test]
    fn bigint_roundtrip(values in prop::collection::vec(prop::num::i64::ANY, 1..10)) {
        // ...
    }
}
```

Note: RESEARCH.md says proptest 1.9 is in dev-dependencies (Cargo.lock shows 1.10.0).
Use `#[proptest(cases = 20)]` or the proptest_config block approach.
</interfaces>
</context>

<feature>
  <name>Property-based tests for binary-read output pipeline</name>
  <files>tests/output_proptest.rs</files>
  <behavior>
    The output_proptest.rs test file validates the binary-read pipeline introduced in Plan 01.

    **Layer 1: Unit PBTs — binary-read helper functions**

    These tests call read_typed_from_vector() on real DuckDB result chunks produced by
    execute_sql_raw() with literal SELECT queries. They do NOT go through the full VTab path.
    They require only Connection::open_in_memory() + the raw C API.

    Properties to verify:
    - BOOLEAN: SELECT true, false, NULL::BOOLEAN → TypedValue::Bool(true), TypedValue::Bool(false), TypedValue::Null
    - TINYINT: boundary values i8::MIN (-128), i8::MAX (127), 0, NULL → correct I8/Null
    - SMALLINT: i16::MIN, i16::MAX → correct I16/Null
    - INTEGER: i32::MIN, i32::MAX → correct I32/Null
    - BIGINT: i64::MIN, i64::MAX → correct I64/Null
    - UTINYINT: 0, 255 → correct U8/Null
    - USMALLINT: 0, 65535 → correct U16/Null
    - UINTEGER: 0, u32::MAX → correct U32/Null
    - UBIGINT: 0, u64::MAX → correct U64/Null
    - FLOAT: -0.0f32, f32::MAX, f32::MIN_POSITIVE → correct F32/Null
    - DOUBLE: -0.0f64, f64::MAX, f64::MIN_POSITIVE → correct F64/Null
    - DATE: '2024-01-15'::DATE → TypedValue::I32(days_since_epoch where days_since_epoch = 19737 for 2024-01-15). This is the key regression: NOT a string parse, direct i32 binary read.
    - TIMESTAMP: TIMESTAMP '2024-01-15 10:30:00' → TypedValue::I64(microseconds_since_epoch). This is the critical TIMESTAMP NULL bug fix — must NOT return TypedValue::Null.
    - TIMESTAMP_S, TIMESTAMP_MS, TIMESTAMP_NS: similar i64 reads
    - VARCHAR: 'hello' → TypedValue::Str("hello")
    - NULL propagation: NULL::INTEGER, NULL::BIGINT, NULL::BOOLEAN → TypedValue::Null

    Proptest-driven unit PBTs (generate arbitrary values, verify no panic):
    - prop: arbitrary i64 values → read as BIGINT → TypedValue::I64(v) where v matches
    - prop: arbitrary i32 values → read as INTEGER → TypedValue::I32(v) where v matches
    - prop: arbitrary bool values → read as BOOLEAN → TypedValue::Bool(v) where v matches
    - prop: arbitrary f64 values (non-NaN, non-Inf) → read as DOUBLE → TypedValue::F64(v) within epsilon

    **Layer 2: Integration PBTs — full roundtrip**

    These tests go through the complete pipeline: create DuckDB table → insert values →
    call execute_sql_raw with the expanded SQL directly → collect via read_typed_from_vector →
    assert output matches input. This tests the full stack without needing the VTab.

    For each type, the test:
    1. Creates an in-memory DuckDB table with the column type
    2. Inserts proptest-generated values
    3. Runs the SQL that expand() would generate for a simple semantic view over that table
    4. Calls read_typed_from_vector on each row of each chunk
    5. Asserts the TypedValue matches the inserted value

    Properties:
    - BIGINT roundtrip: Vec<i64> in, Vec<TypedValue::I64> out, values match
    - INTEGER roundtrip: Vec<i32> in, Vec<TypedValue::I32> out
    - BOOLEAN roundtrip: Vec<bool> in, Vec<TypedValue::Bool> out — BOOLEAN UB fix regression
    - DOUBLE roundtrip: Vec<f64> (finite) in, Vec<TypedValue::F64> out, values within f64 epsilon
    - DATE roundtrip: Vec<i32> (days since epoch) in as DATE values, Vec<TypedValue::I32> out matching days
    - TIMESTAMP roundtrip: Vec<i64> (µs since epoch) in as TIMESTAMP values, Vec<TypedValue::I64> out — TIMESTAMP NULL fix regression
    - VARCHAR roundtrip: Vec<String> in, Vec<TypedValue::Str> out
    - NULL propagation: mix of Some(v) and None values, NULL slots → TypedValue::Null

    DECIMAL integration test (non-proptest, deterministic):
    - INSERT DECIMAL(10,2) values like 350.00, -1.50, 0.00
    - Assert output TypedValue contains the correct integer representation (e.g., for scale=2: 35000, -150, 0)
    - Assert bind() declares DECIMAL(10,2) output type (verified via try_infer_schema on the semantic_view output)

    LIST(BIGINT) integration test (non-proptest, deterministic):
    - INSERT BIGINT[] values like [1,2,3], [], [100]
    - Assert output TypedValue::List with correct inner I64 values
    - Assert bind() declares LIST(BIGINT) output type

    All integration PBTs use `ProptestConfig { cases: 20, .. ProptestConfig::default() }`.
  </behavior>
  <implementation>
    Create `tests/output_proptest.rs` as a new integration test file.

    File structure:
    ```
    // Layer 1 imports
    use duckdb::Connection;
    use libduckdb_sys as ffi;
    use proptest::prelude::*;
    // re-export or pub(crate) access to read_typed_from_vector + execute_sql_raw

    // ── Layer 1: Unit tests (deterministic boundary values) ──────────────────

    mod unit_tests {
        // TIMESTAMP NULL bug regression test
        #[test]
        fn timestamp_returns_non_null() { ... }

        // BOOLEAN UB bug regression test
        #[test]
        fn boolean_reads_correctly() { ... }

        // Boundary values for each scalar type
        #[test]
        fn integer_boundary_values() { ... }

        // DATE binary read regression test
        #[test]
        fn date_binary_read() { ... }
    }

    // ── Layer 1: Proptest unit PBTs (arbitrary values, no panic) ─────────────

    proptest! {
        #![proptest_config(ProptestConfig { cases: 50, .. ProptestConfig::default() })]

        #[test]
        fn bigint_binary_read(v in prop::num::i64::ANY) { ... }

        #[test]
        fn boolean_binary_read(v in any::<bool>()) { ... }

        // etc.
    }

    // ── Layer 2: Integration PBTs (full roundtrip) ───────────────────────────

    proptest! {
        #![proptest_config(ProptestConfig { cases: 20, .. ProptestConfig::default() })]

        #[test]
        fn bigint_column_roundtrip(values in prop::collection::vec(prop::num::i64::ANY, 1..10)) { ... }

        #[test]
        fn boolean_column_roundtrip(values in prop::collection::vec(any::<bool>(), 1..10)) { ... }

        // etc.
    }

    // ── Deterministic integration tests for DECIMAL and LIST ─────────────────

    #[test]
    fn decimal_roundtrip() { ... }

    #[test]
    fn list_bigint_roundtrip() { ... }

    #[test]
    fn null_propagation() { ... }
    ```

    For accessing read_typed_from_vector from the integration test:
    - Either make `read_typed_from_vector` pub(crate) and reference via `semantic_views::query::table_function` (note: this requires the module to be pub)
    - OR test the full pipeline via execute_sql_raw + manual iteration of result chunks using ffi directly in the test (avoids needing to expose internals)
    - Prefer: expose `pub(crate)` on the needed functions and create a `semantic_views::query::test_helpers` module (behind `#[cfg(test)]`) that re-exports them — or simply make the test more black-box by running full SQL and checking row values via the high-level duckdb::Connection::query interface.

    Simplest approach that avoids exposing internals:
    - Use `duckdb::Connection::open_in_memory()` and `conn.prepare("SELECT ...")?.query_map([], |row| ...)` for unit-level type verification
    - For integration roundtrip: create table, insert values, query via `conn.query_row` or `conn.prepare`, assert typed output

    This avoids all the pub(crate) complexity and tests from the user's perspective. The binary read correctness is verified indirectly through the query results.

    Example TIMESTAMP test:
    ```rust
    #[test]
    fn timestamp_returns_non_null() {
        let conn = Connection::open_in_memory().unwrap();
        conn.execute_batch("CREATE TABLE t (ts TIMESTAMP); INSERT INTO t VALUES ('2024-01-15 10:30:00')").unwrap();
        let mut stmt = conn.prepare("SELECT ts FROM t").unwrap();
        let result: Vec<Option<String>> = stmt.query_map([], |row| row.get(0)).unwrap()
            .map(|r| r.unwrap())
            .collect();
        // With binary reads: the value should come back as Some(...)
        // With the old VARCHAR wrapper + parse_typed_from_str bug: would have returned None
        // Since this is an integration test through the VTab, we verify via semantic_view():
        // BUT: we don't have VTab registered here — test via direct SQL instead
        assert_eq!(result.len(), 1);
        assert!(result[0].is_some(), "TIMESTAMP must not be NULL");
        assert!(result[0].as_ref().unwrap().contains("2024-01-15"), "TIMESTAMP value mismatch");
    }
    ```

    For semantic_view() roundtrip tests that go through the full typed output pipeline:
    - Look at how lib.rs exposes extension_entrypoint
    - In tests, call `conn.register_table_function(...)` with the SemanticViewVTab type — this requires duckdb-rs `Connection::register_table_function` method and a `QueryState` instance
    - If this is too complex, the simpler path: write the integration PBTs to test execute_sql_raw + read_typed_from_vector directly via pub(crate) access in a test-only helper module

    Decision (Claude's discretion): Use the two-level approach:
    1. Unit tests (deterministic): use `duckdb::Connection` high-level API to verify type correctness after Plan 01's changes propagate through the semantic_view VTab. Run semantic_view() by registering the function via `semantic_views::register_semantic_view` helper (add a pub fn in lib.rs for test use).
    2. PBTs: test execute_sql_raw + the binary read pipeline directly, treating TypedValue as the contract.

    The simplest correct approach: expose a test-only `pub fn setup_test_db() -> duckdb::Connection` in lib.rs (behind `#[cfg(test)]`) that creates an in-memory DB, registers the SemanticViewVTab, and returns the connection ready for `FROM semantic_view(...)` queries.

    Run: `cargo test output_proptest -- --test-threads=1` for the new test file.
    Full suite: `cargo test`.
  </implementation>
</feature>

<verification>
After implementation:
1. `cargo test output_proptest -- --test-threads=1` — all tests pass
2. `cargo test` — full suite passes (no regressions in expand_proptest.rs or other tests)
3. Check TIMESTAMP test: `cargo test output_proptest::unit_tests::timestamp_returns_non_null` — PASS (was FAIL before Plan 01)
4. Check BOOLEAN test: `cargo test output_proptest::unit_tests::boolean_reads_correctly` — PASS
5. `cargo test output_proptest` timing — should complete in under 60 seconds
</verification>

<success_criteria>
1. `tests/output_proptest.rs` exists with min 150 lines
2. `cargo test output_proptest` passes — all unit + integration PBTs green
3. `cargo test` passes — no regressions in existing tests
4. TIMESTAMP NULL regression covered and passing
5. BOOLEAN UB regression covered and passing
6. DECIMAL and LIST(BIGINT) integration tests passing
7. NULL propagation test passing
</success_criteria>

<output>
After completion, create `.planning/phases/13-type-mapping-and-property-based-tests-for-typed-column-dispatch/13-02-SUMMARY.md`
</output>
