ITADN

[Feature]: yo sis can you create like extend the task scheduled kernal coverage for custom attention operations

#4777Openkds1123001 创建于 1 天前
feature requestneeds-triage
K
kds1123001commented
### Before submitting - [x] I have searched existing issues and this request has not been filed yet. ### Problem bcuz currently the prim ts backend is small like limited in its coverage for some attention (custom) attention configurations and dynamic workload shapes when i try to run ts execution across non standard query length fallback is required and limits concurrency optimization lol ### Requested outcome extend prim ts implementation to support broadend sequence length seamless compactability clear public pythonapi ### Target hardware Other or unknown ### Inference engine vLLM ### Affected model or model family qwen cloud ### Workload and configuration - Data type and quantization: - Batch size or request concurrency: - Sequence lengths or token counts: - Parallelism: TP / EP / DP / PP / disaggregated - Relevant shapes: heads, head dimension, hidden size, experts, top-k, page size, or other - Environment: output of `python -m flashinfer.collect_env` ### Current workaround currently like it is falling back to CUDA graph ### Impact Other ### Acceptance criteria passes unit and integration correctness demonstrates execution stability exposes python api bindings ### Related work, dependencies, or suggested scope _No response_ ### Timing or release need _No response_
1 条评论