ITADN

bug: xinference:v3.0.0,用VLLM启动qwen3.5模型报错

#5219Closedhediyuan 创建于 28 天前
buggpustale
H
hediyuancommented
### System Info / 系統信息 xinference: v3.0.0 用VLLM引擎启动qwen3.5-2b模型失败,报错如下: Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/xinference/core/utils.py", line 98, in wrapped ret = await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/xinference/core/supervisor.py", line 2386, in launch_builtin_model await _launch_model() File "/usr/local/lib/python3.12/dist-packages/xinference/core/supervisor.py", line 2322, in _launch_model raise result File "/usr/local/lib/python3.12/dist-packages/xinference/core/supervisor.py", line 2119, in _launch_one_model subpool_address = await worker_ref.launch_builtin_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/context.py", line 262, in send return self._process_result_message(result) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/context.py", line 111, in _process_result_message raise message.as_instanceof_cause() File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/pool.py", line 689, in send result = await self._run_coro(message.message_id, coro) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/pool.py", line 389, in _run_coro return await coro ^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/xoscar/api.py", line 418, in __on_receive__ return await super().__on_receive__(message) # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "xoscar/core.pyx", line 564, in __on_receive__ File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__ File "xoscar/core.pyx", line 527, in xoscar.core._BaseActor.__on_receive__ File "xoscar/core.pyx", line 532, in xoscar.core._BaseActor.__on_receive__ File "/usr/local/lib/python3.12/dist-packages/xinference/core/utils.py", line 98, in wrapped ret = await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/xinference/core/worker.py", line 3230, in launch_builtin_model model = await asyncio.to_thread( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/threads.py", line 25, in to_thread return await loop.run_in_executor(None, func_call) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/concurrent/futures/thread.py", line 59, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/xinference/model/core.py", line 48, in create_model_instance return create_llm_model_instance( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/xinference/model/llm/core.py", line 310, in create_llm_model_instance llm_cls = check_engine_by_spec_parameters( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/xinference/model/llm/llm_family.py", line 637, in check_engine_by_spec_parameters raise ValueError(f"Model {model_name} cannot be run on engine {model_engine}.") ValueError: [address=0.0.0.0:13717, pid=72] Model qwen3.5 cannot be run on engine vLLM. 2026-07-21T05:54:48.653Z ERROR xinference.api.restful_api pid:1 role:local address:0.0.0.0:9997 node:0a333f17029a [address=0.0.0.0:13717, pid=72] Model qwen3.5 cannot be run on engine vLLM. ------ 点击qwen3.5卡片下的部署按钮后,立马就报错,应该是程序直接拦截了,check_engine_by_spec_parameters这个函数直接rasie ### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece? - [x] docker / docker - [ ] pip install / 通过 pip install 安装 - [ ] installation from source / 从源码安装 ### Version info / 版本信息 V3.0.0 ### The command used to start Xinference / 用以启动 xinference 的命令 xinference launch --model-name qwen3.5 --model-type LLM --n-gpu 1 --replica 1 --enable-thinking --reasoning_content true --n-worker 1 --model-engine vLLM --model-format pytorch --size-in-billions 2 --quantization none --gpu-idx 2 --download_hub modelscope --env VLLM_LOGGING_LEVEL DEBUG --disable-virtual-env --model-uid qwen3.5-2b --max_num_seqs 12 --max_model_len 16384 --language_model_only true --enforce_eager true ### Reproduction / 复现过程 1.启动模型 点击qwen3.5卡片 2.设置好相关参数,选择VLLM engine 3.点击部署模型按钮 ### Expected behavior / 期待表现 启动模型
关闭于 2 天前 3 条评论