ITADN
anentropic/duckdb-semantic-views
anentropic/duckdb-semantic-views · 文件 下载 ZIP
文件最后提交记录最后更新时间
README.md
以下内容由 AI 翻译,如有问题请点此提交 issue 反馈

DuckDB 语义视图

Docs

一个 DuckDB 扩展,允许你一次性定义维度和指标,然后以任意组合进行查询。该扩展为你编写 GROUP BY 和 JOIN 逻辑。

Snowflake Semantic Views 启发,并作为可加载扩展适配于 DuckDB。

工作原理

你在一个或多个表上定义语义视图,声明:

  • Dimensions -- 用于分组的列或表达式(region、category、date_trunc('month', created_at) 等)
  • Metrics -- 聚合函数(sum(amount)count(*) 等)
  • Relationships -- 表之间的 PK/FK 连接路径,仅在查询需要时包含

然后你通过选择所需的维度和指标进行查询。扩展生成 SQL -- SELECT、FROM、JOIN、GROUP BY -- 并由 DuckDB 执行。

快速入门

CREATE TABLE orders (
    id INTEGER, region VARCHAR, category VARCHAR,
    amount DECIMAL(10,2)
);

CREATE SEMANTIC VIEW order_metrics AS
TABLES (
    o AS orders PRIMARY KEY (id)
)
DIMENSIONS (
    o.region AS o.region,
    o.category AS o.category
)
METRICS (
    o.revenue AS sum(o.amount),
    o.order_count AS count(*)
);

-- Pick any combination of dimensions and metrics
SELECT * FROM semantic_view('order_metrics',
    dimensions := ['region', 'category'],
    metrics := ['revenue', 'order_count']
);

-- Dimensions only (distinct values)
SELECT * FROM semantic_view('order_metrics',
    dimensions := ['region']
);

-- Metrics only (grand total)
SELECT * FROM semantic_view('order_metrics',
    metrics := ['revenue']
);

-- WHERE works on the result
SELECT * FROM semantic_view('order_metrics',
    dimensions := ['region'], metrics := ['revenue']
) WHERE region = 'East';

只读数据库: 查询(semantic_viewlist_semantic_viewsdescribe_semantic_view 等)针对以 read_only=True 打开的数据库执行。CREATE / DROP / ALTER SEMANTIC VIEW 需要可写数据库。请参阅 事务性 DDL 及限制 说明页面,了解先引导后重新打开的工作流程。

多表(主键/外键关系)

使用 PRIMARY KEY 和 REFERENCES 定义表之间的关系。仅会连接您请求的维度和指标所需的表。

CREATE TABLE customers (id INTEGER, name VARCHAR, tier VARCHAR);
CREATE TABLE products (id INTEGER, name VARCHAR, category VARCHAR);
CREATE TABLE orders (
    id INTEGER, customer_id INTEGER, product_id INTEGER,
    amount DECIMAL(10,2), region VARCHAR
);

CREATE SEMANTIC VIEW analytics AS
TABLES (
    o AS orders PRIMARY KEY (id),
    c AS customers PRIMARY KEY (id),
    p AS products PRIMARY KEY (id)
)
RELATIONSHIPS (
    order_customer AS o(customer_id) REFERENCES c,
    order_product AS o(product_id) REFERENCES p
)
DIMENSIONS (
    c.customer_name AS c.name,
    p.product_name AS p.name,
    o.region AS o.region
)
METRICS (
    o.revenue AS sum(o.amount),
    o.order_count AS count(*)
);

-- Only customers table is joined (products not needed)
SELECT * FROM semantic_view('analytics',
    dimensions := ['customer_name'],
    metrics := ['revenue']
);

-- Both customers and products tables are joined
SELECT * FROM semantic_view('analytics',
    dimensions := ['customer_name', 'product_name'],
    metrics := ['revenue']
);

使用 explain_semantic_view 查看生成的 SQL:

SELECT * FROM explain_semantic_view('analytics',
    dimensions := ['customer_name'],
    metrics := ['revenue']
);
┌──────────────────────────────────────────────────────────────┐
│                        explain_output                        │
│                           varchar                            │
├──────────────────────────────────────────────────────────────┤
│ -- Semantic View: analytics                                  │
│ -- Dimensions: customer_name                                 │
│ -- Metrics: revenue                                          │
│                                                              │
│ -- Expanded SQL:                                             │
│ SELECT                                                       │
│     c.name AS "customer_name",                               │
│     sum(o.amount) AS "revenue"                               │
│ FROM "orders" AS "o"                                         │
│ LEFT JOIN "customers" AS "c" ON "o"."customer_id" = "c"."id" │
│ GROUP BY                                                     │
│     1                                                        │
│                                                              │
│ -- DuckDB Plan:                                              │
│ ...                                                          │
├──────────────────────────────────────────────────────────────┤
│ 15+ rows                                                     │
└──────────────────────────────────────────────────────────────┘

FACTS(可复用的行级表达式)

将常见的行级计算命名一次,并在指标中引用它们。Facts 在展开时会被内联到指标表达式中。

子句方向: 与 Snowflake 类似,每个条目都是 alias.<logical_name> AS <sql_expression>名称位于 AS 之前,SQL 表达式位于其后。这与普通 SQL expression AS alias 相反。逻辑名称是你查询的对象(facts := ['net_price']),也是 DESCRIBE 返回的列。Fact 可以以其自身的列命名 — FACTS (s.unit_price AS s.unit_price) 定义了一个直通 Fact unit_price。相同的 方向也适用于 DIMENSIONSMETRICS

CREATE SEMANTIC VIEW sales AS
TABLES (
    li AS line_items PRIMARY KEY (id)
)
FACTS (
    li.net_price AS li.extended_price * (1 - li.discount),
    li.tax_amount AS li.net_price * li.tax_rate
)
DIMENSIONS (
    li.region AS li.region
)
METRICS (
    li.total_net AS SUM(li.net_price),
    li.total_tax AS SUM(li.tax_amount)
);

事实可以引用其他事实 -- 扩展会按依赖顺序解析它们。

派生指标(指标组合)

组合不带表前缀的基础指标。扩展会替换底层表达式。

METRICS (
    li.revenue AS SUM(li.net_price),
    li.cost    AS SUM(li.unit_cost),
    profit     AS revenue - cost,
    margin     AS profit / revenue * 100
);

基数与扇形陷阱检测

关系基数是从被引用表上的 PRIMARY KEY / UNIQUE 约束中推断得出的——你无需对其进行标注。对表上已声明键的连接是多对一(或一对一);可能导致聚合结果膨胀的连接会被检测为扇形陷阱,扩展会抛出错误,而不是返回不正确的结果。

RELATIONSHIPS (
    li_to_order AS li(order_id) REFERENCES o,
    order_to_customer AS o(customer_id) REFERENCES c
)

oc 表在 TABLES (... PRIMARY KEY (...)) 中声明了它们的键,因此每个关系的基数取决于目标键。(显式的 ONE TO ONE / ONE TO MANY / MANY TO ONE 注解已在 v0.5.4 中移除,现在会被拒绝。)

角色扮演维度 (USING RELATIONSHIPS)

当同一张表通过多个关系连接时(例如,机场既作为出发地又作为目的地),请在指标上使用 USING 来选择要使用的连接路径。

CREATE SEMANTIC VIEW flight_analytics AS
TABLES (
    f AS flights PRIMARY KEY (flight_id),
    a AS airports PRIMARY KEY (airport_code)
)
RELATIONSHIPS (
    dep_airport AS f(departure_code) REFERENCES a,
    arr_airport AS f(arrival_code) REFERENCES a
)
DIMENSIONS (
    a.city    AS a.city,
    f.carrier AS f.carrier
)
METRICS (
    f.departures USING (dep_airport) AS COUNT(*),
    f.arrivals   USING (arr_airport) AS COUNT(*)
);

如果没有 USING,涉及模糊连接路径的查询将会报错。

DDL 参考

-- Full clause order (RELATIONSHIPS/FACTS optional; at least one of DIMENSIONS or METRICS required)
CREATE SEMANTIC VIEW name AS
  TABLES (...)
  RELATIONSHIPS (...)
  FACTS (...)
  DIMENSIONS (...)
  METRICS (...);

CREATE OR REPLACE SEMANTIC VIEW name AS ...;
CREATE SEMANTIC VIEW IF NOT EXISTS name AS ...;
DROP SEMANTIC VIEW name;
DROP SEMANTIC VIEW IF EXISTS name;
DESCRIBE SEMANTIC VIEW name;
SHOW SEMANTIC VIEWS;

文档

完整文档:anentropic.github.io/duckdb-semantic-views

包含入门教程、DDL 和查询参考、高级功能(FACTS、派生指标、角色扮演维度、扇形陷阱)的使用指南,以及架构说明。

构建

基于 Rust 构建,使用 DuckDB extension template for Rust

您需要:Rust(stable)、just、make、Python 3。

just setup     # one-time: installs dev tools, configures build
just build     # debug build
cargo test     # unit + property-based tests
just test-sql  # SQL logic tests (needs just build first)
just test-all  # everything
just lint      # fmt + clippy + cargo-deny

许可证

MIT