sql: SHOW PARTITIONS persistently fails with unknown schema "[id]" for tables created during concurrent schema drops
C-enhancementT-sql-foundationsC-autosolvetarget-release-26.4.0
## Summary
On v26.2.5, `SHOW PARTITIONS FROM TABLE t` fails with `ERROR: unknown schema
"[<descriptor id>]"` (SQLSTATE 3F000) when table `t` was created while another
session was concurrently dropping schemas. The referenced descriptor id is one
of the concurrently dropped schemas, not anything related to table `t`. The
failure is persistent, not transient: it survives new connections and kept
failing for minutes on a single node and, on a 5 node cluster, for up to the
dropped tables' `gc.ttlseconds` window (an hour in our case) on every gateway
node. Tables created before the churn are unaffected.
Version bracket, same harness on stock single nodes: v25.4.14 and v26.1.8 do
not reproduce (40 to 60 probes each); v26.2.5 and v26.3.0 reproduce on the
first probe. The regression was introduced in the v26.2 release line and is
still present in the latest innovation release. Searching the tracker for
`unknown schema` combined with SHOW PARTITIONS / crdb_internal.partitions
found no existing report.
## Reproduction
Stock single node, insecure, no zone configs, no multi-region, no enterprise
features. Two concurrent SQL sessions:
Session A, in a loop:
```sql
CREATE SCHEMA sc_i;
CREATE TABLE sc_i.t (id INT PRIMARY KEY);
DROP SCHEMA sc_i CASCADE;
```
Session B, in a loop:
```sql
CREATE SCHEMA pr_i;
CREATE TABLE pr_i.t (id INT PRIMARY KEY);
SHOW PARTITIONS FROM TABLE pr_i.t;
```
Session B hits the error within the first one or two iterations, reliably:
```
ERROR: unknown schema "[106]"
SQLSTATE: 3F000
```
A self-contained script (starts a throwaway docker node, runs both loops,
prints the result) is attached; it reproduced on the first probe on every run
we tried against v26.2.5 and never reproduced in 40 to 60 probes against
v25.4.14.
## Observed state while broken
At failure time, `system.descriptor` holds the churned table in state DROP
with `unexposedParentSchemaId` pointing at the schema id in the error message,
and that schema descriptor is already deleted. That orphaned shape also exists
in healthy databases after any schema drop with tables awaiting GC, so it is
not sufficient on its own; the concurrency window during creation of the
probed table is what triggers it. Plan for the failing delegate query shows a
virtual table lookup join into `table_indexes@table_indexes_descriptor_id_idx`;
individually, full scans of `crdb_internal.partitions`, `tables`,
`table_indexes` and `zones` (including the config SQL columns) all succeed
while the delegate query fails.
## Impact
Found via PeerDB's CockroachDB connector, whose mirror validation runs
`SHOW PARTITIONS` per replicated table to compute the effective
`gc.ttlseconds` across partition zone configs. Any workload that creates a
table and inspects its partitions while unrelated schema drops are running
(test suites, multi-tenant provisioning, demo harnesses) can hit this, and the
affected table's SHOW PARTITIONS stays broken long after the drops finish.
## Environment
- Reproduced: v26.2.5 (single node docker and a 5 node cluster behind haproxy,
all gateways affected) and v26.3.0 (single node docker), aarch64 linux
- Not reproduced: v25.4.14 and v26.1.8, same harness
- Introduced in the v26.2 release line per the bracket above
## Self-contained repro script
```bash
#!/bin/bash
# Minimal reproduction: SHOW PARTITIONS fails persistently with
# 'unknown schema "[<id>]"' (SQLSTATE 3F000) for tables created while another
# session concurrently drops schemas. Reproduces on stock v26.2.5 single node
# within one or two probe iterations; does not reproduce on v25.4.14.
#
# Usage: ./crdb-show-partitions-repro.sh [host:port]
# Defaults to a fresh throwaway docker single node on port 26264.
set -u
HOSTPORT="${1:-}"
if [ -z "$HOSTPORT" ]; then
docker rm -f crdb-sp-repro >/dev/null 2>&1
docker run -d --name crdb-sp-repro -p 26264:26257 cockroachdb/cockroach:v26.2.5 start-single-node --insecure >/dev/null
sleep 10
HOSTPORT="localhost:26264"
echo "started throwaway single node cockroachdb/cockroach:v26.2.5 on $HOSTPORT"
fi
C() { cockroach sql --insecure --host "$HOSTPORT" -e "$1" 2>&1; }
C "SELECT version()" | tail -1
# session A: schema churn, plain tables, no zone configs, no MR, nothing fancy
(for i in $(seq 1 60); do
C "CREATE SCHEMA sc_$i; CREATE TABLE sc_$i.t (id INT PRIMARY KEY); DROP SCHEMA sc_$i CASCADE" >/dev/null 2>&1
done) &
CHURN=$!
# session B: create a fresh table and immediately SHOW PARTITIONS on it
for i in $(seq 1 60); do
s="pr_$i"
C "CREATE SCHEMA $s; CREATE TABLE $s.t (id INT PRIMARY KEY)" >/dev/null 2>&1
err=$(C "SHOW PARTITIONS FROM TABLE $s.t" | grep -oE 'unknown schema "\[[0-9]+\]"' | head -1)
if [ -n "$err" ]; then
kill $CHURN 2>/dev/null
echo "REPRODUCED at probe $i: SHOW PARTITIONS FROM TABLE $s.t -> ERROR: $err"
echo "the referenced id is one of session A's dropped schemas"
echo "note: the error persists for this table across new connections;"
echo " rerun 'SHOW PARTITIONS FROM TABLE $s.t' minutes later to confirm"
exit 0
fi
done
wait $CHURN 2>/dev/null
echo "no reproduction in 60 probes"
exit 1
```
Jira issue: CRDB-66970
Epic CRDB-65516
关闭于 5 天前 3 条评论