🚀 Xinference v2.7.0 Release Notes

⚠️

Security Advisory

Versions v2.6.0, v2.6.1, and v2.6.2 were maliciously published to PyPI containing dangerous unauthorized code. These versions have been removed. Please do not install or continue to use them.

✅ Highlights

🧠 New Model Support

  • Qwen3.6
  • GLM-5.1
  • MiniMax-M2.7

🔁 Enhanced Multi-Replica Management

  • Support for dynamic replica removing
  • Parallel execution for multi-replica deletion
  • Further optimized UI deletion interaction experience

🛠 DeepSeek V3.2 Tool Parser Enhancements

  • Added DSML format support
  • Added Plain format support
  • Improved DeepSeek tool call compatibility and stability

🔒 Continuous Stability & Security Improvements

  • Fixed multiple vLLM multi-node distributed issues
  • Fixed race conditions in concurrent virtual environment creation
  • Strengthened tool parser security to avoid potential RCE risks

🌐 Community Edition Updates

📦 Installation Methods

pip install 'xinference==2.7.0'

Docker: Pull the latest image, or update using pip inside the container.

🆕 New Model Support

  • Qwen3.6
  • GLM-5.1
  • MiniMax-M2.7

✨ New Features

  • Support dynamic replica removing
  • Parallelized multi-replica termination operations
  • DeepSeek V3.2 Tool Parser supports DSML / Plain formats
  • Optimized UI configuration cache popups and terminology

🛠 Enhancements

  • Updated xllamacpp to the latest version
  • Optimized aarch64 Docker build process
  • Updated model JSON configurations (Embedding / Rerank / LLM)
  • Refactored DeviceSpec registration mechanism for clearer device adaptation

🐞 Bug Fixes

  • Fixed vLLM multi-node distributed initialization and pipeline parallel inference issues
  • Fixed gemma-4 tool call issues
  • Fixed race condition in concurrent virtual environment creation
  • Fixed supervisor RPC timeout issue
  • Fixed exception caused by missing cpu key in cluster device info
  • Fixed role restriction for the last message in chat completion endpoint
  • Fixed Jina Embeddings v4 torch compatibility issue
  • Fixed embedding / rerank loading failure caused by torchcodec

🔐 Security Fixes

  • Removed eval() and replaced it with a safe implementation to avoid potential RCE risks
  • Fixed pwn-request vulnerability risk in gen_docs workflow

🏢 Enterprise Edition Updates