# What's new in 2.7.0 (2026-04-25) These are the changes in inference v2.7.0. ## New features * FEAT: support replica removing by @leslie2046 in https://github.com/xorbitsai/inference/pull/4784 * feat(ui): optimize configuration cache dialog and terminology by @leslie2046 in https://github.com/xorbitsai/inference/pull/4807 * FEAT: add DeepSeek V3.2 tool parser for DSML format by @amumu96 in https://github.com/xorbitsai/inference/pull/4771 * feat: add Glm4MoeLiteForCausalLM support by @amumu96 in https://github.com/xorbitsai/inference/pull/4835 * feat: parallelize multi-replica terminate_model and improve UI delete UX by @m199369309 in https://github.com/xorbitsai/inference/pull/4825 * FEAT: [model] qwen3.6 support by @llyycchhee in https://github.com/xorbitsai/inference/pull/4831 * FEAT: [model] MiniMax-M2.7 support by @llyycchhee in https://github.com/xorbitsai/inference/pull/4843 * feat(tool_parser): add plain format support to DeepSeek V3.2 tool parser by @amumu96 in https://github.com/xorbitsai/inference/pull/4842 * FEAT: [model] glm-5.1 support by @llyycchhee in https://github.com/xorbitsai/inference/pull/4832 ## Enhancements * ENH: update model "qwen3.5" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/4801 * ENH: update model "DeepSeek-V3.2" JSON by @amumu96 in https://github.com/xorbitsai/inference/pull/4813 * ENH: update models JSON [embedding] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4824 * ENH: update model "qwen3.5" JSON by @qinxuye in https://github.com/xorbitsai/inference/pull/4821 * ENH: update models JSON [embedding, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4841 * ENH: update 2 models JSON ("MiniMax-M2.7", "glm-5.1") by @XprobeBot in https://github.com/xorbitsai/inference/pull/4848 * BLD: update xllamacpp to newest version for docker by @qinxuye in https://github.com/xorbitsai/inference/pull/4819 * BLD: Remove pre-release PyTorch installation from Dockerfile by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4836 * BLD: remove torch related installation in aarch64 dockerfile by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4840 * BLD: remove torchcodec installation in Dockerfile aarch64 by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4844 * BLD: Modify docker build command for aarch64 image by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4853 ## Bug fixes * fix: replace eval() with safe alternatives to prevent RCE in tool parsers by @Ricardo-M-L in https://github.com/xorbitsai/inference/pull/4786 * fix: support JSON object parameters in CLI by @Ricardo-M-L in https://github.com/xorbitsai/inference/pull/4787 * fix: support Jina API task parameters for jina-embeddings-v4 by @Ricardo-M-L in https://github.com/xorbitsai/inference/pull/4788 * fix(ui): handle mixed dict and ChatMessage types in history by @qinxuye in https://github.com/xorbitsai/inference/pull/4814 * fix(vllm): fix gemma-4 tool calls by @llyycchhee in https://github.com/xorbitsai/inference/pull/4815 * fix(docker): unpin torchcodec to fix 503 error on reranker/embedding model load by @FlintyLemming in https://github.com/xorbitsai/inference/pull/4817 * fix: handle missing 'cpu' key in get_cluster_device_info to prevent KeyError 500 by @m199369309 in https://github.com/xorbitsai/inference/pull/4822 * fix: venv concurrent creation race, cold-start lock dir, and jina-embeddings-v4 torch mismatch by @m199369309 in https://github.com/xorbitsai/inference/pull/4823 * fix: dynamic CUDA version check for extra_index_url by @Gmgge in https://github.com/xorbitsai/inference/pull/4820 * fix: vLLM multi-node distributed init and pipeline parallel inference by @amumu96 in https://github.com/xorbitsai/inference/pull/4834 * fix: venv torchvision alignment, supervisor RPC timeouts, get_model flood protection, replica pre-check, and safe log handler by @m199369309 in https://github.com/xorbitsai/inference/pull/4839 * fix: remove last message role restriction in chat completion endpoint by @amumu96 in https://github.com/xorbitsai/inference/pull/4833 * fix(security): prevent pwn-request vulnerability in gen_docs workflow by @qinxuye in https://github.com/xorbitsai/inference/pull/4850 ## Documentation * DOC: add v2.5.0 release notes by @qinxuye in https://github.com/xorbitsai/inference/pull/4800 ## Others * refactor(device_utils): replace if/elif chains with DeviceSpec registry by @amumu96 in https://github.com/xorbitsai/inference/pull/4846 ## New Contributors * @Ricardo-M-L made their first contribution in https://github.com/xorbitsai/inference/pull/4786 * @FlintyLemming made their first contribution in https://github.com/xorbitsai/inference/pull/4817 * @m199369309 made their first contribution in https://github.com/xorbitsai/inference/pull/4822 **Full Changelog**: https://github.com/xorbitsai/inference/compare/v2.5.0...v2.7.0

# What's new in 2.6.0 (2026-04-21) These are the changes in inference v2.6.0 ## New Features Just bug fixes :)

# What's new in 2.1.0 (2026-02-14) These are the changes in inference v2.1.0. ## New features * FEAT: [model] GLM-4.7 support by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4565 * FEAT: [model] MinerU2.5-2509-1.2B removed by @OliverBryant in https://github.com/xorbitsai/inference/pull/4568 * FEAT: [model] GLM-4.7-Flash support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4578 * FEAT: [model] Qwen3-ASR-0.6B support by @leslie2046 in https://github.com/xorbitsai/inference/pull/4579 * FEAT: [model] Qwen3-ASR-1.7B support by @leslie2046 in https://github.com/xorbitsai/inference/pull/4580 * FEAT: added support qwen3-asr models by @leslie2046 in https://github.com/xorbitsai/inference/pull/4581 * FEAT: [model] MinerU2.5-2509-1.2B support by @GaoLeiA in https://github.com/xorbitsai/inference/pull/4569 * FEAT: [model] FLUX.2-klein-4B support by @lazariv in https://github.com/xorbitsai/inference/pull/4602 * FEAT: [model] FLUX.2-klein-9B support by @lazariv in https://github.com/xorbitsai/inference/pull/4603 * FEAT: Add support for FLUX.2-Klein-9B and -4B models by @lazariv in https://github.com/xorbitsai/inference/pull/4596 ## Enhancements * ENH: update model "DeepSeek-V3.2" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4563 * ENH: update model "DeepSeek-V3.2-Exp" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4567 * ENH: update models JSON [image] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4606 * BLD: constrain setuptools<82 in Docker images by @qinxuye in https://github.com/xorbitsai/inference/pull/4607 * REF: extract Pydantic request schemas from restful_api.py into xinference/api/schemas/ by @amumu96 in https://github.com/xorbitsai/inference/pull/4598 * REF: extract route registration into domain-specific routers/ by @amumu96 in https://github.com/xorbitsai/inference/pull/4600 ## Bug fixes * BUG: vllm embedding model error by @OliverBryant in https://github.com/xorbitsai/inference/pull/4562 * BUG: vllm reranker score error by @OliverBryant in https://github.com/xorbitsai/inference/pull/4573 * BUG: handle async tokenizer in vllm core by @ace-xc in https://github.com/xorbitsai/inference/pull/4577 * BUG: vllm reranker model gpu release error by @OliverBryant in https://github.com/xorbitsai/inference/pull/4575 ## Documentation * DOC: add v2.0.0 release by @qinxuye in https://github.com/xorbitsai/inference/pull/4566 ## Others * BUG：setuptools CI error by @OliverBryant in https://github.com/xorbitsai/inference/pull/4595 ## New Contributors * @ace-xc made their first contribution in https://github.com/xorbitsai/inference/pull/4577 * @GaoLeiA made their first contribution in https://github.com/xorbitsai/inference/pull/4569 * @lazariv made their first contribution in https://github.com/xorbitsai/inference/pull/4602 **Full Changelog**: https://github.com/xorbitsai/inference/compare/v2.0.0...v2.1.0

v1.17.1 is a hotfix version of v1.17.0 **Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.17.0...v1.17.1

# What's new in 1.15.0 (2025-12-13) These are the changes in inference v1.15.0. ## New features * FEAT: added more detailed instructions for engine unavailability. by @OliverBryant in https://github.com/xorbitsai/inference/pull/4308 * FEAT: [model] Z-Image-Turbo support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4333 * FEAT: [model] DeepSeek-V3.2 support by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4344 * FEAT: [model] PaddleOCR-VL support by @leslie2046 in https://github.com/xorbitsai/inference/pull/4354 * FEAT: add llama_cpp json schema output by @OliverBryant in https://github.com/xorbitsai/inference/pull/4282 * FEAT: PaddleOCR-VL implementation by @leslie2046 in https://github.com/xorbitsai/inference/pull/4304 * FEAT: multi replicas on a single GPU && add launch strategy by @OliverBryant in https://github.com/xorbitsai/inference/pull/4358 ## Enhancements * ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4343 * ENH: update model "MiniMax-M2" JSON by @XprobeBot in https://github.com/xorbitsai/inference/pull/4342 * ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4349 * ENH: support lauching with --device cpu by @hubutui in https://github.com/xorbitsai/inference/pull/4352 * ENH: add glm-4.5 tool calls support && vllm StructuredOutputsParams support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4357 ## Bug fixes * BUG: fix manage cache models missing by @OliverBryant in https://github.com/xorbitsai/inference/pull/4329 * BUG: [llm, vllm]: support ignore eos by @ZhikaiGuo960110 in https://github.com/xorbitsai/inference/pull/4332 * BUG: Multimodal settings for video parameters are not taking effect. by @OliverBryant in https://github.com/xorbitsai/inference/pull/4338 * BUG: Soft links cannot be completely deleted by @OliverBryant in https://github.com/xorbitsai/inference/pull/4337 * BUG: Packages with identical names in virtual environments error by @OliverBryant in https://github.com/xorbitsai/inference/pull/4348 * BUG: Fix typo in xinference/deploy/docker/Dockerfile.cu128 by @hubutui in https://github.com/xorbitsai/inference/pull/4350 * BUG: custom embedding model register fail by @OliverBryant in https://github.com/xorbitsai/inference/pull/4335 * BUG: [UI] fix the bug in the copy function. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4355 * BUG: [UI] control Select dropdown width to prevent it from becoming too wide. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4356 ## Documentation * DOC: add new models and v1.14.0 release notes by @qinxuye in https://github.com/xorbitsai/inference/pull/4305 ## Others * Fixed- workflow Vulnerability by @barakharyati in https://github.com/xorbitsai/inference/pull/4328 * CHORE: add i18n for replica details by @leslie2046 in https://github.com/xorbitsai/inference/pull/4306 ## New Contributors * @barakharyati made their first contribution in https://github.com/xorbitsai/inference/pull/4328 * @ZhikaiGuo960110 made their first contribution in https://github.com/xorbitsai/inference/pull/4332 * @hubutui made their first contribution in https://github.com/xorbitsai/inference/pull/4350 **Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.14.0...v1.15.0

# What's new in 1.14.0 (2025-11-30) These are the changes in inference v1.14.0. ## New features * FEAT: add vLLM 0.11.1+ compatibility with v1 executor support by @amumu96 in https://github.com/xorbitsai/inference/pull/4252 * FEAT: [virtualenv] New v3 spec and list/delete virtual env APIs by @OliverBryant in https://github.com/xorbitsai/inference/pull/4254 * FEAT: [model] HunyuanOCR support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4290 * FEAT: Add support of rerank model for llamacpp by @harryzwh in https://github.com/xorbitsai/inference/pull/4227 * FEAT: show reason why engines not available by @OliverBryant in https://github.com/xorbitsai/inference/pull/4261 * FEAT: Parallel startup model, add tooltips for startup progress, and p… by @leslie2046 in https://github.com/xorbitsai/inference/pull/4268 ## Enhancements * BLD: fix model ui launch error with gradio 6.x by @OliverBryant in https://github.com/xorbitsai/inference/pull/4289 * BLD: add pr auto run gen_docs workflow. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4260 * BLD: gen docs pr modify by @OliverBryant in https://github.com/xorbitsai/inference/pull/4294 * BLD: gen doc modify v2 by @OliverBryant in https://github.com/xorbitsai/inference/pull/4296 * BLD: gen docs pr modify v3 by @OliverBryant in https://github.com/xorbitsai/inference/pull/4297 * BLD: auto-run gen_docs.py from doc/source by @yiboyasss in https://github.com/xorbitsai/inference/pull/4300 * BLD: remove [skip ci] from auto docs commit by @yiboyasss in https://github.com/xorbitsai/inference/pull/4301 ## Bug fixes * BUG: Compat with xllamacpp 0.2.5+ by @codingl2k1 in https://github.com/xorbitsai/inference/pull/4270 * BUG: add download_hubs for cluster by @OliverBryant in https://github.com/xorbitsai/inference/pull/4273 * BUG: sometimes cannot select gpu in CPU and GPU hybrid cluster by @leslie2046 in https://github.com/xorbitsai/inference/pull/4280 ## Documentation * DOC: added v1.13.0 release notes by @qinxuye in https://github.com/xorbitsai/inference/pull/4250 * DOC: update gen_docs by @qinxuye in https://github.com/xorbitsai/inference/pull/4302 ## Others * CHORE: expand stale and close time by @qinxuye in https://github.com/xorbitsai/inference/pull/4253 * chore: sync models JSON [audio, embedding, image, llm, rerank, video] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4258 * chore: sync models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4272 * chore: sync model "Qwen3-Reranker-0.6B" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4277 * chore: sync model "bge-reranker-v2-m3" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4276 * chore: sync model "Qwen3-Reranker-4B" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4278 * chore: sync model "Qwen3-Reranker-8B" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4279 * chore: sync model "qwen3" JSON by @XprobeBot in https://github.com/xorbitsai/inference/pull/4287 * chore: sync models JSON [rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4284 * chore: sync model "FLUX.1-dev" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4293 * chore: sync model "FLUX.2-dev" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4292 * chore: sync models JSON [image] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4303 **Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.13.0...v1.14.0

# What's new in 1.12.0 (2025-11-02) These are the changes in inference v1.12.0. ## New features * FEAT: [model] support jina-reranker-v3 by @llyycchhee in https://github.com/xorbitsai/inference/pull/4156 * FEAT: [model] qwen3-omni by @qinxuye in https://github.com/xorbitsai/inference/pull/4137 * FEAT: xinference python 3.13 support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4164 * FEAT: add OCR gradio UI by @OliverBryant in https://github.com/xorbitsai/inference/pull/4185 * FEAT: [model] DeepSeek-OCR by @OliverBryant in https://github.com/xorbitsai/inference/pull/4187 ## Enhancements * ENH: adding lightning support for qwen-image-edit-2509 by @qinxuye in https://github.com/xorbitsai/inference/pull/4151 * BLD: torchaudio 2.9 introduces the breaking change in torchaudio.save by @qiulang in https://github.com/xorbitsai/inference/pull/4178 * BLD: fix setup.cfg for python 3.12 and fix dockerfile by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4192 * BLD: fix Dockerfile.cpu by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4195 * REF: Modified the batch lock logic by @OliverBryant in https://github.com/xorbitsai/inference/pull/4162 ## Bug fixes * BUG: repair qwen3 model transformers random characters by @OliverBryant in https://github.com/xorbitsai/inference/pull/4148 * BUG: [UI] resolve progress bar display issue. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4150 * BUG: fix IndexTTS2 on transformes 4.57.1 by @OliverBryant in https://github.com/xorbitsai/inference/pull/4158 * BUG: fix error when xinference run on docker with oath2 by @OliverBryant in https://github.com/xorbitsai/inference/pull/4161 * BUG: fix qwen3-vl launch error by @amumu96 in https://github.com/xorbitsai/inference/pull/4190 ## Documentation * DOC: add release notes doc by @qinxuye in https://github.com/xorbitsai/inference/pull/4157 * DOC: Add PyPI mirror configuration guide for audio package installation by @qiulang in https://github.com/xorbitsai/inference/pull/4177 ## Others * BLD：fix transformers version in cu128 dockerfile by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4152 * chore: sync models JSON [image, llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4149 * chore: sync models JSON [rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4159 * chore: sync models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4160 * chore: sync models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4171 * chore: sync models JSON [image] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4186 * chore: sync models JSON [embedding, image] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4188 * chore: sync models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4191 **Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.11.0...v1.12.0

# What's new in 1.11.0.post1 (2025-10-20) These are the changes in inference v1.11.0.post1. ## Bug fixes * BUG: repair qwen3 model transformers random characters by @OliverBryant in https://github.com/xorbitsai/inference/pull/4148 * BUG: [UI] resolve progress bar display issue. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4150 ## Others * BLD：fix transformers version in cu128 dockerfile by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4152 **Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.11.0...v1.11.0.post1