[Bug] Qwen3.5 + FA2 + neat_packing can create invalid cu_seqlens and crash FLA GDN kernel

# Qwen3.5 + FA2 + neat_packing can create invalid cu_seqlens and crash FLA GDN kernel ### Reminder - [x] I have read the above rules and searched the existing issues. ### System Info ```text - `llamafactory` version: 0.9.5.dev0 - Platform: Linux-5.15.0-171-generic-x86_64-with-glibc2.35 - Python version: 3.11.15 - PyTorch version: 2.7.0+cu128 - Transformers version: 5.2.0 - Datasets version: 3.2.0 - Accelerate version: 1.11.0 - PEFT version: 0.18.1 - TRL version: 0.24.0 - DeepSpeed version: 0.16.4 - Git commit: 958323aeb49177419f5a9bd6380fd9bbd24fabd7 - flash-attn version: 2.7.4.post1 - flash-linear-attention version: 0.5.0 - Triton version: 3.3.0 - GPU: 8x NVIDIA A800-SXM4-80GB ``` ### Reproduction I am testing Qwen3.5-4B VL SFT with frozen base model, FA2, and neat packing. The Qwen3.5 patch from recent main is active; the log prints: ```text Patched Qwen3.5 decoder forward to support cu_seqlens input only patch when do training. ``` Relevant config: ```yaml model_name_or_path: /models/common/Qwen3.5-4B trust_remote_code: true deepspeed: examples/deepspeed/ds_z0_config.json flash_attn: fa2 stage: sft do_train: true finetuning_type: freeze freeze_trainable_layers: 4 freeze_trainable_modules: all freeze_vision_tower: true freeze_multi_modal_projector: true freeze_language_model: false template: qwen3_5_nothink cutoff_len: 4096 neat_packing: true packing: true max_samples: 64 per_device_train_batch_size: 1 gradient_accumulation_steps: 16 bf16: true max_steps: 1 val_size: 0.0 eval_strategy: "no" ``` The dataset is a local ShareGPT-format mixed SFT dataset containing text, image, and video examples. Each source dataset was filtered with `cutoff_len=4096`. I minimized the trigger by replacing one source at a time: ```text PASS: image + video + UltraChat text PASS: image + video + COIG text PASS: image + COIG text + UltraChat text PASS: video + COIG text + UltraChat text FAIL: image + video + COIG text + UltraChat text PASS: same four datasets with neat_packing=false FAIL: same four datasets with neat_packing=true ``` The first training forward fails inside the patched Qwen3.5 GDN varlen path: ```text [rank2]: File "src/llamafactory/model/patcher.py", line 208, in _patch_gdn_forward [rank2]: core_attn_out, _ = chunk_gated_delta_rule( [rank2]: File ".../site-packages/fla/ops/gated_delta_rule/chunk.py", line 497, in chunk_gated_delta_rule [rank2]: o, final_state = ChunkGatedDeltaRuleFunction.apply( [rank2]: File ".../site-packages/fla/ops/gated_delta_rule/chunk.py", line 282, in forward [rank2]: g, o, A, final_state, initial_state, g_input = chunk_gated_delta_rule_fwd( [rank2]: File ".../site-packages/fla/ops/gated_delta_rule/chunk.py", line 91, in chunk_gated_delta_rule_fwd [rank2]: h, v_new, final_state = chunk_gated_delta_rule_fwd_h( [rank2]: File ".../site-packages/fla/ops/common/chunk_delta_h.py", line 699, in chunk_gated_delta_rule_fwd_h [rank2]: chunk_gated_delta_rule_fwd_kernel_h_blockdim64[grid]( [rank2]: RuntimeError: Triton Error [CUDA]: invalid argument ``` This is not OOM. Peak memory before failure is around 29-30 GiB per GPU on 80 GiB A800s. I added a temporary local debug log before `chunk_gated_delta_rule(...)` in `_patch_gdn_forward`. In the failing run, one rank receives an over-length packed sample and very suspicious `cu_seqlens`: ```text Qwen3.5 GDN debug: rank=1 layer=0 hidden=(1, 4168, 2560) query=(1, 4168, 32, 128) key=(1, 4168, 32, 128) value=(1, 4168, 32, 128) position_ids=(1, 4168) cu_seqlens=[0, 1359, 2648, 3927, 4054, 4055, 4056, 4057, 4058, ..., 4168] ``` So even though `cutoff_len=4096`, the tensor passed to FLA is length 4168, and the tail of `cu_seqlens` contains many consecutive 1-token segments. This seems related to the `has_dummy_image and has_packing` path in the multimodal collator. In pure-text packed microbatches under a VL model, the collator injects a dummy image. With neat packing, the padded/dummy tail appears to remain in `position_ids` and later becomes many fake sequence boundaries for `prepare_fa_kwargs_from_position_ids`, causing invalid `cu_seqlens` for the FLA GDN kernel. I also tried lowering `cutoff_len` to 3900. That did not solve it; it exposed a nearby collator mismatch instead: ```text ValueError: Merged position_ids shape mismatch: got torch.Size([3, 1, 3900]), expected (3, 1, 3904). ``` ### Others Related PRs/issues I checked: - #10436 adds the Qwen3.5 `neat_packing` patch and varlen linear attention path. - #10448 fixes a Qwen3.5 FA2 illegal memory access issue. - #8919 is an older FA2 + neat_packing shape-mismatch report. This issue happens after #10436/#10448 are present and after the Qwen3.5 patch is applied. I cannot share the private dataset, but the minimized behavior above suggests the root cause is not one malformed sample. It is triggered when mixed multimodal data produces a pure-text packed microbatch under Qwen3.5 VL + FA2 + neat_packing, causing dummy-image padding/position IDs to leak into `cu_seqlens`.