ITADN

OGA export generating an invalid model when a lm_head unquantized model is exported

#2185Openuday610 创建于 2026-05-27
quantization
U
uday610commented
**Describe the bug** OGA export generating an invalid model when a lm_head unquantized model is exported **To Reproduce** Steps to reproduce the behavior: 1. Please access the quantized model from here https://huggingface.co/amd/granite-4.0-1b-awq-grp128-exclude_lmhead_quark 2. Create OGA environment, I have tested from 0.12.1 to latest 0.13.2. I have been using `transformers==4.57.6` and `torch==2.12.0` 3. Export using this command ``` python -m onnxruntime_genai.models.builder ` -i "granite_awq_lmhead_exclude" ` -o "granite_oga_int4_lmhead_exclude" ` -p int4 ` -e cpu ` --extra_options exclude_embeds=true int4_nodes_to_exclude=/lm_head/MatMul ``` 4. See the message of invalid model ``` 2026-05-26 13:21:43,041 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /model/constant_nodes/FLOAT/0.125 ... 2026-05-26 13:21:43,041 onnxruntime.quantization.matmul_nbits_quantizer [INFO] - skip to quantize /lm_head/Mul ... 2026-05-26 13:21:43,057 onnx_ir.serde [WARNING] - Input 'lm_head.MatMul.weight_Q4G32' of node '/model/embed_tokens/Reshape' (::Reshape:) cannot be found in any scope. The model is invalid but we will still create a new input for the node (current depth: 1) 2026-05-26 13:21:43,057 onnx_ir.serde [WARNING] - Input 'lm_head.MatMul.weight_scale' of node '/model/embed_tokens/GatherBlockQuantized' (com.microsoft::GatherBlockQuantized:) cannot be found in any scope. The model is invalid but we will still create a new input for the node (current depth: 1) ```
0 条评论