ITADN

[Bug]: AuthenticationFailed when querying multiple Unity catalog tables in same Datafusion context

#4425Openhjohnss6 创建于 2026-05-03
binding/rustunity-catalogawaiting-feedback
H
hjohnss6commented
### What happened? When using the catalog-unity crate to register a catalog in Datafusion, a query with multiple tables, or consecutive queries in the same context, fails. ### Expected behavior A single Datafusion context can query multiple tables from Unity. ### Operating System macOS Linux ### Binding Rust ### Bindings Version 0.16.0 ### Steps to reproduce 1. Have a Unity catalog catalog in Databricks (enable external access) 2. Register the catalog in Datafusion 3. Query table1 (stored in `abfss://cont@acc.dfs.core.windows.net/path1`) 4. Query table2 (stored in `abfss://cont@acc.dfs.core.windows.net/path2`) (OBS: same storage account and container) ```rust let ctx = SessionContext::new(); ctx.register_catalog("new_catalog", Arc::new(new_catalog)); let df = ctx .sql("SELECT * FROM new_catalog.schema.table1 LIMIT 100") .await .expect("run query"); df.show().await.expect("show results"); let df = ctx .sql("SELECT * FROM new_catalog.schema.table2 LIMIT 100") .await .expect("run query"); df.show().await.expect("show results"); ``` ### Relevant logs ```bash ParquetError(General("Failed to fetch metadata for file xxxx: Parquet error: External: The operation lacked the necessary privileges to complete for path xxxx: Error performing GET REDACTED in 64.81975ms - Server returned non-2xx status code: 403 Forbidden: \u{feff}<?xml version=\"1.0\" encoding=\"utf-8\"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.Message><AuthenticationErrorDetail>Signature did not match. String to sign used was xxxx </AuthenticationErrorDetail></Error>")) ``` ### Investigations (and speculation) The temporary storage credentials created at: https://github.com/delta-io/delta-rs/blob/54f43b6fc68d5db7030832b379591d9486105c61/crates/catalog-unity/src/lib.rs#L838 Gives a vended credential for Azure storage including the path to the table root. (A SAS for a path) The credential is then used to create a table, including storage connection here: https://github.com/delta-io/delta-rs/blob/54f43b6fc68d5db7030832b379591d9486105c61/crates/catalog-unity/src/datafusion.rs#L248 The registration is likely to be on a storage root level, mentioned here: https://github.com/delta-io/delta-rs/blob/54f43b6fc68d5db7030832b379591d9486105c61/crates/core/src/delta_datafusion/table_provider.rs#L804 So, when registering a second table, with same storage root, the old credential is reused. ### What does work - Restart SessionContext between the queries - Queries on tables stored in different storage accounts/containers
2 条评论