ITADN

[Bug][create-release] ARM64 cu129 flashinfer-jit-cache wheel exceeds the 2 GiB GitHub Release asset limit

#4519OpenMaryWang-Nvidia 创建于 15 天前
bugneeds-triage
M
MaryWang-Nvidiacommented
### Before submitting - [x] I have searched existing issues and did not find a solution or an existing report. ### Environment ```text N/A — this is a release-packaging issue, not a runtime/import failure in a Python environment. Affected build environment: - Workflow: Nightly Release #347 https://github.com/flashinfer-ai/flashinfer/actions/runs/31661197699 - Head SHA: 8f9ad2000db7fa9970b397086296b0c6e6461b02 - Build matrix: CUDA 12.9, aarch64 - Builder image: pytorch/manylinuxaarch64-builder:cuda12.9 - Artifact: flashinfer_jit_cache-0.6.18.dev20260813+cu129-cp39-abi3-manylinux_2_28_aarch64.whl `python -m flashinfer.collect_env` is not applicable because the failure occurs while publishing the wheel, before an end-user Python runtime exists. ``` ### Bug description Nightly Release #347 produced an oversized ARM64 CUDA 12.9 JIT-cache wheel. The package-size report measured: - Current size: 2194.6 MB - Hard limit: 2048 MB / 2 GiB - Excess: 146.6 MB - Change from baseline: +300.8 MB (+15.9%) GitHub Release assets must be smaller than 2 GiB. Therefore this wheel cannot be published through `gh release upload`, which prevents a complete nightly release. Expected behavior is that every release wheel stays below the GitHub Release per-asset limit, or that the package is split/published through a compatible channel. ### Minimal reproduction ```python This is a CI/release packaging regression rather than a runtime Python bug. 1. Check out the nightly head: git clone --recursive https://github.com/flashinfer-ai/flashinfer.git cd flashinfer git checkout 8f9ad2000db7fa9970b397086296b0c6e6461b02 2. Run the JIT-cache build with the nightly ARM64 CUDA 12.9 configuration: export CUDA_MAJOR=12 export CUDA_MINOR=9 export FLASHINFER_LOCAL_VERSION=cu129 export FLASHINFER_CUDA_ARCH_LIST="7.5 8.0 8.9 9.0a 10.0a 10.3a 12.0f 12.1a" docker run --rm \ -v "$PWD:/workspace" \ -w /workspace \ -e CUDA_VERSION=12.9 \ -e CUDA_MAJOR \ -e CUDA_MINOR \ -e FLASHINFER_LOCAL_VERSION \ -e ARCH=aarch64 \ -e FLASHINFER_CUDA_ARCH_LIST \ pytorch/manylinuxaarch64-builder:cuda12.9 \ bash /workspace/scripts/build_flashinfer_jit_cache_whl.sh 3. Measure the generated wheel: ls -lh flashinfer-jit-cache/dist/*.whl The resulting cu129/aarch64 wheel is ~2194.6 MB, exceeding the 2 GiB GitHub Release asset limit. Uploading it with `gh release upload` is expected to fail with a size-limit validation error. ``` ### Error messages / logs ```text ``` ### Additional context ## Root-cause evidence **Primary cause:** - [PR #3684](https://github.com/flashinfer-ai/flashinfer/pull/3684) (tip merge of Nightly 347) extended nightly/release FLASHINFER_CUDA_ARCH_LIST with 12.1a (SM121) only for ARM64 CUDA 12.9 and 13.0. x86_64 lists and CUDA 12.8 are unchanged. The JIT-cache wheel therefore packages an extra full set of SM121 kernels on the two aarch64 variants that regress. **Secondary / partial contributors:** - [PR #4318](https://github.com/flashinfer-ai/flashinfer/pull/4318) / [PR #4069](https://github.com/flashinfer-ai/flashinfer/pull/4069): plausible for some tracked .so growth (e.g. fused_moe, topk), but cannot explain the ARM64-only multi-hundred-MB jump. - [PR #4457](https://github.com/flashinfer-ai/flashinfer/pull/4457): Docker CI tag bump may add shared rebuild noise on the order of the x86 same-CUDA delta (~35–45 MB), not the ARM64-extra bulk.
0 条评论