ITADN

阿里云盘中的om文件未能初始化获取到输出

#6OpenHDZsuper 创建于 2025-02-03
H
HDZsupercommented
Ascend310B1,CANN7.0 请问为什么在阿里云盘中使用的tiny-llama在推理时没能够正确读取模型并获得输出呢? > (inference) root@orangepiaipro-20t:~/Desktop/lab3/ascend-llm/inference# python main.py --model /root/Desktop/tinyLlama-1.1B-Chat/tiny-llama-seq-1-key-1024-int8.om --h f-dir /root/Desktop/tinyLlama-1.1B-Chat --engine acl --sampling greedy --cli None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used. InferenceConfig(tokenizer='/root/Desktop/tinyLlama-1.1B-Chat', hf_model_dir='/root/Desktop/tinyLlama-1.1B-Chat', sampling_method='greedy', sampling_value=10, temperature=0.7, max_length=512, max_input_len=1, session_type='acl', device=0, prompt=[{'role': 'user', 'content': 'Hey there I am a human that would like to have a conversation with you.'}, {'role': 'assistant', 'content': 'Sure, I am happy to answer your questions'}, {'role': 'user', 'content': 'Great, I insist that we take turns.'}, {'role': 'assistant', 'content': 'I agree, we should take turns.'}], model='/root/Desktop/tinyLlama-1.1B-Chat/tiny-llama-seq-1-key-1024-int8.om', kvcache_method='sliding-window', kvcache_fixsize=True, head_len=32, recent_len=32, evict_len=64, n_layer=22, format='huggingface-tensor', max_cache_size=256, head_num=4, num_kv_group=8, head_dim=64, hidden_dim=2048, dtype='float16', model_type='tiny-llama') return code is 500002, detail: load model return code is 545008, detail: get model desc [] Traceback (most recent call last): File "/root/Desktop/lab3/ascend-llm/inference/main.py", line 94, in <module> engine = LlamaInterface(cfg) File "/root/Desktop/lab3/ascend-llm/inference/inference.py", line 17, in __init__ self.session=Session.fromConfig(config) File "/root/Desktop/lab3/ascend-llm/inference/session.py", line 20, in fromConfig return AclSession(config) File "/root/Desktop/lab3/ascend-llm/inference/session.py", line 76, in __init__ self.input_ids,_,_,self.kvCache.kvCache = self.model.getInputs() ValueError: not enough values to unpack (expected 4, got 0) ls /root/Desktop/tinyLlama-1.1B-Chat README.md configuration.json gitattributes special_tokens_map.json tokenizer.json tokenizer_config.json config.json generation_config.json quantize_config.json tiny-llama-seq-1-key-1024-int8.om tokenizer.model
0 条评论