bug: xinference:v3.0.0,用VLLM启动qwen3.5模型报错
buggpustale
### System Info / 系統信息
xinference: v3.0.0
用VLLM引擎启动qwen3.5-2b模型失败,报错如下:
Traceback (most recent call last):
File "/usr/local/lib/python3.12/dist-packages/xinference/core/utils.py", line 98, in wrapped
ret = await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xinference/core/supervisor.py", line 2386, in launch_builtin_model
await _launch_model()
File "/usr/local/lib/python3.12/dist-packages/xinference/core/supervisor.py", line 2322, in _launch_model
raise result
File "/usr/local/lib/python3.12/dist-packages/xinference/core/supervisor.py", line 2119, in _launch_one_model
subpool_address = await worker_ref.launch_builtin_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/context.py", line 262, in send
return self._process_result_message(result)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/context.py", line 111, in _process_result_message
raise message.as_instanceof_cause()
File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/pool.py", line 689, in send
result = await self._run_coro(message.message_id, coro)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xoscar/backends/pool.py", line 389, in _run_coro
return await coro
^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xoscar/api.py", line 418, in __on_receive__
return await super().__on_receive__(message) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 564, in __on_receive__
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
File "xoscar/core.pyx", line 527, in xoscar.core._BaseActor.__on_receive__
File "xoscar/core.pyx", line 532, in xoscar.core._BaseActor.__on_receive__
File "/usr/local/lib/python3.12/dist-packages/xinference/core/utils.py", line 98, in wrapped
ret = await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xinference/core/worker.py", line 3230, in launch_builtin_model
model = await asyncio.to_thread(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/threads.py", line 25, in to_thread
return await loop.run_in_executor(None, func_call)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/thread.py", line 59, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xinference/model/core.py", line 48, in create_model_instance
return create_llm_model_instance(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xinference/model/llm/core.py", line 310, in create_llm_model_instance
llm_cls = check_engine_by_spec_parameters(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/xinference/model/llm/llm_family.py", line 637, in check_engine_by_spec_parameters
raise ValueError(f"Model {model_name} cannot be run on engine {model_engine}.")
ValueError: [address=0.0.0.0:13717, pid=72] Model qwen3.5 cannot be run on engine vLLM.
2026-07-21T05:54:48.653Z ERROR xinference.api.restful_api pid:1 role:local address:0.0.0.0:9997 node:0a333f17029a [address=0.0.0.0:13717, pid=72] Model qwen3.5 cannot be run on engine vLLM.
------
点击qwen3.5卡片下的部署按钮后,立马就报错,应该是程序直接拦截了,check_engine_by_spec_parameters这个函数直接rasie
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
- [x] docker / docker
- [ ] pip install / 通过 pip install 安装
- [ ] installation from source / 从源码安装
### Version info / 版本信息
V3.0.0
### The command used to start Xinference / 用以启动 xinference 的命令
xinference launch --model-name qwen3.5 --model-type LLM --n-gpu 1 --replica 1 --enable-thinking --reasoning_content true --n-worker 1 --model-engine vLLM --model-format pytorch --size-in-billions 2 --quantization none --gpu-idx 2 --download_hub modelscope --env VLLM_LOGGING_LEVEL DEBUG --disable-virtual-env --model-uid qwen3.5-2b --max_num_seqs 12 --max_model_len 16384 --language_model_only true --enforce_eager true
### Reproduction / 复现过程
1.启动模型 点击qwen3.5卡片
2.设置好相关参数,选择VLLM engine
3.点击部署模型按钮
### Expected behavior / 期待表现
启动模型
关闭于 2 天前 3 条评论