ITADN

Issue with OGA granite-4.0-1b model response

#2020Closedsanket131192 创建于 2026-03-12
S
sanket131192commented
OGA CPU granite-4.0-1b model generates repetitive response, whereas, converted PyTorch model generates good response. Model link - [ibm-granite/granite-4.0-1b · Hugging Face](https://huggingface.co/ibm-granite/granite-4.0-1b) The above model is traditional dense LLM. However, it is packaged as "GraniteMoeHybridForCausalLM". So, the model is converted to "GraniteforCausalLM" using the script [ibm-granite/granite-4.0-micro · Why GraniteMoeHybridForCausalLM?](https://huggingface.co/ibm-granite/granite-4.0-micro/discussions/4#690cf543c1c1d9887828d545) **** The above script gives a converted PyTorch Model which is used as input PyTorch model while performing OGA export. **** Note - onnxruntime-genai version == 0.12.1 Command used to perform OGA export - python -m onnxruntime_genai.models.builder -i "path\to\model\" -o "output\path\" -p fp32 -e cpu
关闭于 2026-03-13 2 条评论