Refactor: Import SimpleDB and SimpleTable from simple-data-analysis-core, extend with extra methods

## Context A new stripped-down library `simple-data-analysis-core` has been created with only `@duckdb/node-api` as its dependency. All core data methods are already implemented there. This repo (`simple-data-analysis`) currently has local copies of `SimpleDB` and `SimpleTable` that are **identical** to the core versions, plus 15 additional methods that depend on external packages. The goal is to import `SimpleDB` and `SimpleTable` from `simple-data-analysis-core` and extend them here, keeping all existing functionality and tests intact. **Dependency:** Requires `simple-data-analysis-core` ≥ version with `tableClass` property ([issue #12](https://github.com/nshiab/simple-data-analysis-core/issues/12)). --- ## Current State | Aspect | `simple-data-analysis-core` | `simple-data-analysis` (current) | |--------|----------------------------|----------------------------------| | **SimpleDB** | ✅ Complete + `tableClass` property | ✅ Identical (local copy) | | **SimpleTable** | ✅ Core data methods | ✅ Core + 15 extra methods | | **Dependencies** | `@duckdb/node-api`, `@std/assert` | 9 dependencies (AI, dataviz, google, format, plot, ollama, zod, d3-dsv, arrow) | | **Tests** | ~130 test files | ~157 test files | --- ## Extra Methods in Current (Not in Core) These 15 methods depend on external packages and will live in the extended `SimpleTable`: | Method | Dependency | GitHub Issue | |--------|-----------|--------------| | `aiRowByRow` | `@nshiab/journalism-ai`, `ollama` | #1185 | | `aiRowByRowPool` | `@nshiab/journalism-ai`, `ollama` | #1185 | | `aiEmbeddings` | `@nshiab/journalism-ai`, `ollama` | #1185 | | `aiVectorSimilarity` | `@nshiab/journalism-ai`, `ollama` | #1185 | | `aiQuery` | `@nshiab/journalism-ai`, `ollama` | #1185 | | `aiRAG` | `@nshiab/journalism-ai`, `ollama` | #1185 | | `hybridSearch` | AI methods above | #1185 | | `loadSheet` | `@nshiab/journalism-google` | #1187 | | `toSheet` | `@nshiab/journalism-google` | #1187 | | `logBarChart` | `@nshiab/journalism-dataviz`, `@observablehq/plot` | #1186 | | `logDotChart` | `@nshiab/journalism-dataviz`, `@observablehq/plot` | #1186 | | `logLineChart` | `@nshiab/journalism-dataviz`, `@observablehq/plot` | #1186 | | `logHistogram` | `@nshiab/journalism-dataviz` | #1186 | | `writeChart` | `@observablehq/plot` | #1186 | | `writeMap` | `@nshiab/journalism-dataviz` | #1186 | --- ## Implementation Plan ### Phase 1: Restructure Class Files (Import from Core) **1.1 Update \`src/class/Simple.ts\`** The `Simple` base class is not exported from core, so it is kept locally unchanged. **1.2 Rewrite \`src/class/SimpleDB.ts\`** Extend core's `SimpleDB` and set `tableClass` to our extended `SimpleTable`: \`\`\`ts import { SimpleDB as SimpleDBCore } from "@nshiab/simple-data-analysis-core"; import SimpleTable from "./SimpleTable.ts"; export default class SimpleDB extends SimpleDBCore { constructor(options?: SimpleDBOptions) { super(options); this.tableClass = SimpleTable; } } \`\`\` No method overrides needed — the core's `tableClass` property ensures all table creation uses our extended class. **1.3 Rewrite \`src/class/SimpleTable.ts\`** Strip the file down to only the 15 extra methods, extending core's `SimpleTable`: \`\`\`ts import { SimpleTable as SimpleTableCore } from "@nshiab/simple-data-analysis-core"; export default class SimpleTable extends SimpleTableCore { // aiRowByRow, aiRowByRowPool, aiEmbeddings, aiVectorSimilarity, // hybridSearch, aiRAG, aiQuery, loadSheet, toSheet, // writeChart, writeMap, logLineChart, logDotChart, logBarChart, logHistogram } \`\`\` No method overrides needed — the core delegates table creation to \`this.sdb.newTable()\`, which respects \`tableClass\`. **1.4 Update \`src/index.ts\`** No changes needed — still exports \`SimpleDB\` and \`SimpleTable\`. --- ### Phase 2: Handle Extra Methods and Dependencies **2.1 AI Methods (Issue #1185)** Keep: \`aiRowByRow.ts\`, \`aiRowByRowPool.ts\`, \`aiEmbeddings.ts\`, \`aiVectorSimilarity.ts\`, \`aiQuery.ts\`, \`aiRAG.ts\`, \`hybridSearch.ts\` Keep: \`tryAI.ts\`, \`tryEmbedding.ts\` helpers Keep dependency: \`@nshiab/journalism-ai\`, \`ollama\` **2.2 Google Sheets Methods (Issue #1187)** Keep: \`loadSheet\` and \`toSheet\` methods Keep dependency: \`@nshiab/journalism-google\` **2.3 Charting Methods (Issue #1186)** Keep: \`logHistogram.ts\` method file Keep charting methods: \`logBarChart\`, \`logDotChart\`, \`logLineChart\`, \`writeChart\`, \`writeMap\` Keep dependency: \`@nshiab/journalism-dataviz\`, \`@observablehq/plot\` **2.4 Other Dependencies to Keep** - \`@nshiab/journalism-format\` — used for \`camelCase\`, \`formatNumber\`, \`prettyDuration\` - \`d3-dsv\` — used for \`csvFormat\` - \`zod\` — used in AI methods - \`apache-arrow\` — keep for now --- ### Phase 3: deno.json Updates **3.1 Remove local file reference, add JSR reference:** \`\`\`json "imports": { "@nshiab/simple-data-analysis-core": "jsr:@nshiab/simple-data-analysis-core@^0.0.6", // ... existing imports } \`\`\` **3.2 Keep all current dependencies** — they're all needed for the extended methods. --- ### Phase 4: Keep Tests Intact **4.1 All existing tests remain unchanged** - All 157 test files in \`test/unit/methods/\` continue to work - The inherited methods from core are already tested by core's test suite - Tests for the 15 extra methods continue to run against the extended SimpleTable **4.2 No test migration needed** — since we're extending (not replacing), the API is identical. --- ### Phase 5: Validation **5.1 Run checks:** \`\`\`bash deno fmt --check deno lint deno check src/index.ts deno test -A --fail-fast \`\`\` --- ## Summary of File Changes | File | Action | |------|--------| | \`src/class/Simple.ts\` | Kept locally (not exported from core) | | \`src/class/SimpleDB.ts\` | Rewritten: thin wrapper (~20 lines), extends core | | \`src/class/SimpleTable.ts\` | Stripped to ~1500 lines of extra methods only, extends core | | \`deno.json\` | Add core package JSR reference | | \`src/index.ts\` | No changes | | \`test/unit/\` | No changes — all tests preserved | --- ## Risk Mitigation 1. **Backwards compatible** — public API doesn't change; all methods remain available 2. **Tests act as safety net** — all 157 existing tests must pass 3. **Minimal override surface** — zero method overrides needed (thanks to \`tableClass\` pattern) 4. **Dependencies untouched** — all 9 external dependencies remain for the extended methods