[ci] [dask] Dask tests on macOS are very expensive
maintenancedask
## Description
Opening this to track work around reducing the time it takes to run the Dask tests.
On my macOS (M2 macbook Pro w/ 16 GB memory), as of the latest commit (https://github.com/lightgbm-org/LightGBM/commit/8e7b3d51977dc90a7abb4cae6c0069a126430d02), I see the following:
```console
$ pytest --durations=50 --durations-min=0.25 tests/python_package_test/test_dask.py
==== 328 passed, 12 skipped, 3 xfailed, 45 xpassed, 1524 warnings in 252.83s (0:04:12) ===
```
<details><summary>timings for test cases that take more than 1 second (click me)</summary>
```consle
$ pytest --durations=50 --durations-min=0.25 tests/python_package_test/test_dask.py
3.23s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-dataframe-with-categorical-ranking]
2.97s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-dataframe-with-categorical-ranking]
2.84s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-dataframe-ranking]
2.73s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-dataframe-ranking]
2.59s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-array-ranking]
2.48s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-array-ranking]
2.35s call tests/python_package_test/test_dask.py::test_init_score[array-ranking]
2.26s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-scipy_csr_matrix-binary-classification]
2.15s call tests/python_package_test/test_dask.py::test_network_params_not_required_but_respected_if_given[ranking]
2.12s call tests/python_package_test/test_dask.py::test_ranker[data-goss-None-array]
2.12s call tests/python_package_test/test_dask.py::test_ranker[data-dart-None-array]
2.11s call tests/python_package_test/test_dask.py::test_ranker[voting-goss-None-array]
2.11s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-scipy_csr_matrix-binary-classification]
2.08s call tests/python_package_test/test_dask.py::test_ranker[voting-dart-None-array]
2.05s call tests/python_package_test/test_dask.py::test_ranker[voting-gbdt-None-array]
1.98s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-array-binary-classification]
1.90s call tests/python_package_test/test_dask.py::test_training_works_if_client_not_provided_or_set_after_construction[ranking]
1.89s call tests/python_package_test/test_dask.py::test_ranker[data-gbdt-None-array]
1.89s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-scipy_csr_matrix-multiclass-classification]
1.83s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-scipy_csr_matrix-regression]
1.80s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-dataframe-with-categorical-binary-classification]
1.72s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-array-binary-classification]
1.69s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-dataframe-with-categorical-binary-classification]
1.63s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-dataframe-binary-classification]
1.60s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-scipy_csr_matrix-multiclass-classification]
1.59s call tests/python_package_test/test_dask.py::test_training_succeeds_even_if_some_workers_do_not_have_any_data[array-ranking]
1.57s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-dataframe-binary-classification]
1.56s call tests/python_package_test/test_dask.py::test_model_and_local_version_are_picklable_whether_or_not_client_set_explicitly[True-ranking-joblib]
1.56s call tests/python_package_test/test_dask.py::test_ranker[voting-goss-None-dataframe-with-categorical]
1.55s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-scipy_csr_matrix-regression]
1.55s call tests/python_package_test/test_dask.py::test_ranker[voting-rf-None-dataframe]
1.50s call tests/python_package_test/test_dask.py::test_ranker[data-rf-None-dataframe]
1.49s call tests/python_package_test/test_dask.py::test_model_and_local_version_are_picklable_whether_or_not_client_set_explicitly[False-ranking-cloudpickle]
1.48s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-dataframe-with-categorical-regression]
1.48s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-dataframe-with-categorical-regression]
1.47s call tests/python_package_test/test_dask.py::test_model_and_local_version_are_picklable_whether_or_not_client_set_explicitly[True-binary-classification-pickle]
1.40s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-dataframe-with-categorical-multiclass-classification]
1.39s call tests/python_package_test/test_dask.py::test_classifier[data-gbdt-binary-classification-array]
1.39s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-dataframe-multiclass-classification]
1.38s call tests/python_package_test/test_dask.py::test_classifier[voting-dart-multiclass-classification-dataframe-with-categorical]
1.38s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-dataframe-regression]
1.37s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-array-regression]
1.35s call tests/python_package_test/test_dask.py::test_training_succeeds_even_if_some_workers_do_not_have_any_data[array-binary-classification]
1.34s call tests/python_package_test/test_dask.py::test_classifier[voting-gbdt-multiclass-classification-dataframe-with-categorical]
1.34s call tests/python_package_test/test_dask.py::test_model_and_local_version_are_picklable_whether_or_not_client_set_explicitly[True-ranking-pickle]
1.33s call tests/python_package_test/test_dask.py::test_model_and_local_version_are_picklable_whether_or_not_client_set_explicitly[True-ranking-cloudpickle]
1.32s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[specified-eval_sizes0-array-multiclass-classification]
1.31s call tests/python_package_test/test_dask.py::test_model_and_local_version_are_picklable_whether_or_not_client_set_explicitly[False-ranking-joblib]
1.31s call tests/python_package_test/test_dask.py::test_model_and_local_version_are_picklable_whether_or_not_client_set_explicitly[False-ranking-pickle]
1.30s call tests/python_package_test/test_dask.py::test_eval_set_no_early_stopping[None-eval_sizes0-dataframe-regression]
```
</details>
<details><summary>output of 'pip list' (click me)</summary>
```text
Package Version
----------------- -----------
build 1.5.0
cffi 2.0.0
click 8.4.0
cloudpickle 3.1.2
contourpy 1.3.3
cycler 0.12.1
dask 2026.3.0
distributed 2026.3.0
fonttools 4.63.0
fsspec 2026.4.0
iniconfig 2.3.0
Jinja2 3.1.6
joblib 1.5.3
kiwisolver 1.5.0
lightgbm 4.6.0.99
locket 1.0.0
MarkupSafe 3.0.3
matplotlib 3.10.9
msgpack 1.1.2
numpy 2.4.6
packaging 26.2
pandas 3.0.3
partd 1.4.2
pillow 12.2.0
pip 24.3.1
pluggy 1.6.0
polars 1.40.1
polars-runtime-32 1.40.1
psutil 7.2.2
pyarrow 24.0.0
pycparser 3.0
Pygments 2.20.0
pyparsing 3.3.2
pyproject_hooks 1.2.0
pytest 9.0.3
python-dateutil 2.9.0.post0
PyYAML 6.0.3
scikit-learn 1.8.0
scipy 1.17.1
six 1.17.0
sortedcontainers 2.4.0
tblib 3.2.2
threadpoolctl 3.6.0
toolz 1.1.0
tornado 6.5.5
urllib3 2.7.0
zict 3.0.0
```
</details>
## Benefits of this work
* reduces CI time generally (#7292), which should help with throughput during busy periods in the repo
* reduces time-per-commit spend on macOS CI runners from GitHub Actions, one of this projects' scarcest resources (#7223)
## Approach
TBD (investigating, just opening to track)
## Notes
N/A
0 条评论