Xinference v2.4.0 Release Notes

Release Highlights

Introduced OpenTelemetry as a native observability platform, providing deep insights into inference workloads.

Added support for aarch64 images. Seamlessly deploy on ARM CPU machines like the NVIDIA DGX Spark.

The Enterprise Edition brings significant optimizations and experience enhancements tailored for version 1.0.

Huge congratulations to @leslie2046 for being promoted to maintainer! Thank you for your continued contributions.

• Introducing OTEL by @leslie2046 in #4666
• [UI] Add Xagent link by @yiboyasss in #4693
• [UI] Remove featured/all toggle and prioritize featured models by @yiboyasss in #4694
• Support vLLM v0.18.0 by @llyycchhee in #4718
• Add GPU load metrics by @leslie2046 in #4712
• Upgrade base image to v0.17.1 & add support for aarch64 version images by @zwt-1234 in #4726
• Fix aarch64 build in CI by @zwt-1234 in #4735

• Update model "qwen3.5" JSON by @qinxuye (#4689) & @llyycchhee (#4707)
• Update models JSON [llm] by @XprobeBot in #4710 & #4713
• Adapt normalize param of vLLM >0.16.0 for embedding models by @la1ty in #4729
• Requirements dependency version adjustments by @zwt-1234 in #4736, #4737, #4738
• Parallelize supervisor model registration listing by @leslie2046 in #4690

• Fix async client FormData handling and response lifecycle issues by @qinxuye in #4687
• Fix MLX backend accumulating intermediate generation steps into final output by @nasircsms in #4617
• Inject parent site-packages into child venv via .pth file by @nasircsms in #4692
• Fix launch multi GPU qwen3.5 error by @llyycchhee in #4700
• Fix tool calls for qwen3.5 by @llyycchhee in #4703 & #4709
• Fix qwen3.5 reasoning parse by @llyycchhee in #4719
• Support XML-like tool call format in non-streaming mode for qwen3.5 by @amumu96 in #4715
• Fix WebUI crash when gpu_utilization is none by @leslie2046 in #4728