ITADN

Bug: llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35'

#910Opentoohamster 创建于 2026-03-14
bughigh severity
T
toohamstercommented
### Contact Details 449211678@qq.com ### What happened? The qwen3.5 quantization model I downloaded doesn't work. I downloaded it from here: https://huggingface.co/unsloth/Qwen3.5-0.8B-GGUF/ ### Version llamafile v0.9.3 ### What operating system are you seeing the problem on? Mac ### Relevant log output ```shell note: if you have an AMD or NVIDIA GPU then you need to pass -ngl 9999 to enable GPU offloading llama_model_loader: loaded meta data with 46 key-value pairs and 320 tensors from ./Qwen3.5-0.8B-Q4_K_M.gguf (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = qwen35 llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.name str = Qwen3.5-0.8B llama_model_loader: - kv 3: general.basename str = Qwen3.5-0.8B llama_model_loader: - kv 4: general.quantized_by str = Unsloth llama_model_loader: - kv 5: general.size_label str = 0.8B llama_model_loader: - kv 6: general.license str = apache-2.0 llama_model_loader: - kv 7: general.license.link str = https://huggingface.co/Qwen/Qwen3.5-0... llama_model_loader: - kv 8: general.repo_url str = https://huggingface.co/unsloth llama_model_loader: - kv 9: general.base_model.count u32 = 1 llama_model_loader: - kv 10: general.base_model.0.name str = Qwen3.5 0.8B llama_model_loader: - kv 11: general.base_model.0.organization str = Qwen llama_model_loader: - kv 12: general.base_model.0.repo_url str = https://huggingface.co/Qwen/Qwen3.5-0.8B llama_model_loader: - kv 13: general.tags arr[str,1] = ["image-text-to-text"] llama_model_loader: - kv 14: qwen35.block_count u32 = 24 llama_model_loader: - kv 15: qwen35.context_length u32 = 262144 llama_model_loader: - kv 16: qwen35.embedding_length u32 = 1024 llama_model_loader: - kv 17: qwen35.feed_forward_length u32 = 3584 llama_model_loader: - kv 18: qwen35.attention.head_count u32 = 8 llama_model_loader: - kv 19: qwen35.attention.head_count_kv u32 = 2 llama_model_loader: - kv 20: qwen35.rope.dimension_sections arr[i32,4] = [11, 11, 10, 0] llama_model_loader: - kv 21: qwen35.rope.freq_base f32 = 10000000.000000 llama_model_loader: - kv 22: qwen35.attention.layer_norm_rms_epsilon f32 = 0.000001 llama_model_loader: - kv 23: qwen35.attention.key_length u32 = 256 llama_model_loader: - kv 24: qwen35.attention.value_length u32 = 256 llama_model_loader: - kv 25: qwen35.ssm.conv_kernel u32 = 4 llama_model_loader: - kv 26: qwen35.ssm.state_size u32 = 128 llama_model_loader: - kv 27: qwen35.ssm.group_count u32 = 16 llama_model_loader: - kv 28: qwen35.ssm.time_step_rank u32 = 16 llama_model_loader: - kv 29: qwen35.ssm.inner_size u32 = 2048 llama_model_loader: - kv 30: qwen35.full_attention_interval u32 = 4 llama_model_loader: - kv 31: qwen35.rope.dimension_count u32 = 64 llama_model_loader: - kv 32: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 33: tokenizer.ggml.pre str = qwen35 llama_model_loader: - kv 34: tokenizer.ggml.tokens arr[str,248320] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 35: tokenizer.ggml.token_type arr[i32,248320] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 36: tokenizer.ggml.merges arr[str,247587] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... llama_model_loader: - kv 37: tokenizer.ggml.eos_token_id u32 = 248046 llama_model_loader: - kv 38: tokenizer.ggml.padding_token_id u32 = 248055 llama_model_loader: - kv 39: tokenizer.chat_template str = {%- set image_count = namespace(value... llama_model_loader: - kv 40: general.quantization_version u32 = 2 llama_model_loader: - kv 41: general.file_type u32 = 15 llama_model_loader: - kv 42: quantize.imatrix.file str = Qwen3.5-0.8B-GGUF/imatrix_unsloth.gguf llama_model_loader: - kv 43: quantize.imatrix.dataset str = unsloth_calibration_Qwen3.5-0.8B.txt llama_model_loader: - kv 44: quantize.imatrix.entries_count u32 = 186 llama_model_loader: - kv 45: quantize.imatrix.chunks_count u32 = 80 llama_model_loader: - type f32: 133 tensors llama_model_loader: - type q8_0: 36 tensors llama_model_loader: - type q4_K: 98 tensors llama_model_loader: - type q5_K: 36 tensors llama_model_loader: - type q6_K: 17 tensors llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35' llama_load_model_from_file: failed to load model ./Qwen3.5-0.8B-Q4_K_M.gguf: failed to load model ```
7 条评论