ITADN

epic: Speed up slow Playwright tests in CI

#2308Openschloerke 创建于 2026-07-03
choreepictesting
S
schloerkecommented
## Summary Analysis of CI run [28639206793](https://github.com/posit-dev/py-shiny/actions/runs/28639206793) (last fully-green `Run tests` on `main`) shows that while most Playwright tests complete in ~1–2 seconds, a handful of tests take 10–55+ seconds each. Durations were collected from the pytest `--durations=6 --durations-min=5.0` reports across all 20 Python 3.13 chromium/webkit playwright jobs (`playwright-shiny`, `playwright-examples`, `playwright-ai` × 4 shards). These slow tests dominate shard wall-clock time and are the main lever for speeding up CI. > **Caveat:** pytest only reports the top 6 durations ≥5s per job, so this is a worst-offenders list, not an exhaustive census. Firefox jobs were not sampled but should follow the same pattern. ## Extreme outliers (20s+ per test) | Test | Duration | |---|---| | `tests/playwright/shiny/bugs/0696-resolve-id/test_0696_resolve_id.py::test_module_support` | **56.5s** webkit / 17.3s chromium | | `tests/playwright/ai_generated_apps/bookmark/input_slider/test_input_slider_express_bookmarking.py::test_slider_parameters[app-express.py]` | **~48s** (both browsers) | | `tests/playwright/shiny/inputs/test_input_slider.py::test_input_slider_kitchen` | 37s chromium / 31s webkit | | `tests/playwright/shiny/inputs/test_inputs_update.py::test_updates` | 25s chromium / 22s webkit | ## Whole-file offender: `input_slider` app suite Every test in `tests/playwright/shiny/inputs/input_slider/test_input_slider_app.py` takes 14–20s (`test_slider_range`, `test_slider_loop`, `test_slider_date_format`, `test_slider_drag_range_disabled`, `test_slider_custom_format`) — roughly **80–90s per browser** for the file. Slider tests dominate overall because drags are simulated with many small mouse moves and waits. ## 10–20s tier - `data_frame/html_columns/test_html_columns.py::test_validate_html_columns` — ~18s each for pandas and polars variants - `data_frame/row_selection/test_row_selection.py::test_row_selection` — 16s webkit - `data_frame/df_methods/test_df_methods.py::test_dataframe_methods` — 15s call + **13s setup** - `examples/test_shiny_create.py` — `dashboard-tips`, `basic-navigation`, `basic-sidebar`, `dashboard` template tests run 12–20s each (each runs `shiny create` then boots the app) - `examples/test_external_templates.py` — `stock-app`, `regularization`, `model-scoring`, `dashboard-tips`, `nba-dashboard` at 8–14s - `ai_generated_apps/core_express/navset_pill/test_navset_pill_core_express.py::test_navset_pill_demo` — 12–13s per app variant - `examples/test_examples.py` — `busy_indicators` (13s), `model-score` (12s) ## Slow setup phases (app startup, not the test) Several tests show 12–13s in the pytest `setup` phase — the Shiny app process booting and importing pandas/plotting libraries before the browser connects: - `data_frame/df_methods/test_df_methods.py` - `data_frame/example/test_data_frame.py::test_single_selection` - `busy_indicators/test_busy_indicators.py` ## Tasks - [ ] #2309 — Investigate `test_0696_resolve_id.py::test_module_support` (56s on webkit vs 17s chromium) - [ ] #2310 — Reduce ~48s AI-generated slider bookmarking test (`test_slider_parameters`) - [ ] #2311 — Reduce slider drag-simulation cost (`test_input_slider_kitchen` + `input_slider/test_input_slider_app.py`) - [ ] #2312 — Investigate `test_inputs_update.py::test_updates` (22–25s) - [ ] #2313 — Reduce data frame test durations (`html_columns`, `row_selection`, `df_methods`) - [ ] #2314 — Reduce 12–13s app-startup (setup phase) time (data_frame/busy_indicators apps) - [ ] #2315 — Trim app-boot cost in `test_shiny_create` / `test_external_templates`
0 条评论