ITADN

Deeply nested async fns in mem_wal::scanner produce oversized futures and near-limit trait-solver chains

#8416Openwjones127 创建于 22 天前
performance
W
wjones127commented
## Summary The point-lookup path in `mem_wal::scanner` stacks six `async fn` frames, three of which are `#[instrument]`-annotated. On Rust 1.97 each `async` block gained additional `MaybeDangling`/`ManuallyDrop` wrapper layers, and each `#[instrument]` contributes three type layers of its own (`Instrumented` → `ManuallyDrop` → `MaybeDangling`). The result is an auto-trait obligation chain deep enough that a downstream crate awaiting these futures overflows the default `recursion_limit` of 128 and fails to compile. Observed against `v10.1.0-beta.2` with Rust 1.97.1. The downstream fix was `#![recursion_limit = "256"]`, which works but pushes the cost onto every consumer rather than addressing the depth here. ## The chain The obligation starts in the io_uring reader's moka cache types and then walks every nested async frame: ``` overflow evaluating the requirement `Arc<uring::reader::CacheKey>: Sync` within `moka::common::concurrent::KeyHash<CacheKey>` within `moka::common::concurrent::entry_info::EntryInfo<CacheKey>` required for `triomphe::Arc<EntryInfo<CacheKey>>` to implement `Sync` within `ValueEntry<CacheKey, CachedReaderData>` ... within async fn body dataset/mem_wal/scanner/point_lookup.rs:435 within async block dataset/mem_wal/scanner/point_lookup.rs:350 within `MaybeDangling<{async block@LsmPointLookupPlanner::lookup::{closure#0}::{closure#0}}>` within `ManuallyDrop<...>` within `Instrumented<...>` <- #[instrument] within async fn body point_lookup.rs:355 within async block point_lookup.rs:455 within `MaybeDangling<...>` / `ManuallyDrop<...>` / `Instrumented<...>` <- #[instrument] within async fn body point_lookup.rs:460 within async fn body point_lookup.rs:590 within async fn body scanner/builder.rs:687 within async fn body scanner/builder.rs:527 within async fn body scanner/builder.rs:769 ... downstream async frames ... help: consider increasing the recursion limit by adding a `#![recursion_limit = "256"]` attribute ``` The `#[instrument]` frames that compound it: | Function | Location | | --- | --- | | `lsm_point_lookup` | `dataset/mem_wal/scanner/point_lookup.rs:195` | | `lsm_lookup` | `dataset/mem_wal/scanner/point_lookup.rs:350` | | `lsm_lookup_many` | `dataset/mem_wal/scanner/point_lookup.rs:455` | That is roughly nine layers of pure wrapper types before any actual state machine content, in a single call path. ## Why boxing at the boundary doesn't help The chain already terminates in a `Pin<Box<dyn Future + Send>>` coercion downstream (`#[async_trait]`). A box only truncates the chain *above* it — everything below still has to be proven at the box site, which is where the overflow occurs. To actually shorten the proof, the box has to sit *below* the deep nest, i.e. inside this call path. ## Proposals **1. Add `clippy::large_futures` as a regression guard.** It measures future size rather than nesting depth, but the two correlate here, and it is prescriptive — it names the await site and suggests the exact `Box::pin`: ``` warning: large future with a size of 4098 bytes 14 | big().await; | ^^^^^ help: consider `Box::pin` on it: `Box::pin(big())` ``` `clippy.toml` already exists in the repo, so this is a one-line addition: ```toml future-size-threshold = 16384 # clippy's default; tighten as the tail is cleaned up ``` The lint is in the `pedantic` group, so it also needs enabling — either `-W clippy::large_futures` on the existing CI clippy invocation (`cargo clippy --profile ci --locked --features $ALL_FEATURES --all-targets -- -D warnings`) or `#![warn(clippy::large_futures)]` in the crate roots. **2. `Box::pin` the worst offenders in this path**, so the auto-trait chain is cut inside Lance rather than leaving each consumer to raise its own recursion limit. **3. Reduce per-frame `#[instrument]` layering.** Instrumenting an already-erased future (`Box::pin(fut).instrument(span)`) or instrumenting only at the outer boundary keeps the spans without compounding three type layers per frame. ## Finding the offenders Ranked inventory of async state machine sizes: ```bash RUSTFLAGS="-Zprint-type-sizes" cargo +nightly build -p lance 2>&1 \ | grep -A2 "^print-type-size type: .*{async" | sort -u ``` Note that recursion *depth* has no equivalent lint, and it cannot be canaried by lowering the limit in CI: `-Z min-recursion-limit` computes `max(flag, crate_limit)`, so it can only raise the limit, never lower it. Future size is the only practical proxy.
0 条评论