Release Notes · August 2026

Xinference 3.2

Per-replica scaling and placement, logprobs across vLLM and sglang, PDF OCR with DeepDoc document parsing, automatic download-source detection, and a wave of new models — a focused upgrade on the 3.1 foundation.

🧮 Per-Replica Scaling 📈 logprobs Everywhere 📄 OCR + DeepDoc 🧩 New Models

What's New

Highlights

🧮

Per-Replica Scaling & Placement

Scale running models by replica, place each replica on a chosen worker and GPU, and see per-replica runtime resources.

  • • Scale running models by replica
  • • Per-replica worker & GPU placement
  • • Per-replica runtime resources
📈

logprobs Everywhere

logprobs and top_logprobs are now surfaced across both the vLLM and sglang engines.

  • • /v1/chat/completions logprobs
  • • sglang completions & chat logprobs
  • • Consistent scoring output
📄

OCR & Document Parsing

PDF input for the OCR API, a built-in DeepDoc image model, and a document-parsing pipeline exposed as task="parse".

  • • PDF input for the OCR API
  • • Built-in DeepDoc (ragflow) model
  • • Parse pipeline & OvisOCR2
🧩

New Models & Engines

Day-one support for MiniMax-H3 video, Ling 3.0, minimax-m3, indextts 2.5, and more, with automatic hub detection.

  • • MiniMax-H3, Ling 3.0, minimax-m3
  • • jina-reranker-v3.5, R3 embed/rerank
  • • Auto HF to ModelScope detection

Open Source

Community Edition

Installation

pip

pip install 'xinference==3.2.0'

Docker

docker pull xprobe/xinference:latest

✨ New Features

  • ▸ /v1/chat/completions logprobs
  • ▸ Deploy R3-embedding-0.6b & R3-rerank-0.6b models
  • ▸ jina-reranker-v3.5 with native listwise API
  • ▸ PDF input support for the OCR API
  • ▸ sglang logprobs in /v1/completions & /v1/chat/completions
  • ▸ DeepDoc (ragflow) OCR as a built-in image model
  • ▸ minimax-m3 model support
  • ▸ Per-replica worker & GPU placement
  • ▸ WebUI per-replica placement controls
  • ▸ Scale running models by replica
  • ▸ WebUI running-model replica scaling
  • ▸ Expose & display per-replica runtime resources
  • ▸ DeepDoc document-parsing pipeline as task="parse"
  • ▸ CAMPPlus speaker embedding support
  • ▸ FireRed Image Edit 1.1 support
  • ▸ Wan2.2 Animate 2 video models
  • ▸ Auto-detect download hub between Hugging Face & ModelScope
  • ▸ MiniMax-H3 video support
  • ▸ Ling 3.0 model support
  • ▸ Enhanced download progress details
  • ▸ indextts 2.5
  • ▸ OvisOCR2 model support

🛠 Enhancements

  • ▸ Updated model configs (embedding / rerank / image / llm / audio / video)
  • ▸ Searchable dropdowns in Audit Center text filters
  • ▸ Page navigation & deep pagination in log and audit centers
  • ▸ Whole-result debounce cache for list_models (supervisor)

🐞 Bug Fixes

  • ▸ Avoid Set-changed-during-iteration crash & gc overhead under concurrent downloads
  • ▸ Add anthropic SDK to core dependencies for route registration
  • ▸ Use Xorbits/deepdoc mirror for the DeepDoc Hugging Face source
  • ▸ WebUI: ignore stale launch model requests
  • ▸ Fix audit center text filters returning no results
  • ▸ Preserve Jina v3 allocator env on worker recovery
  • ▸ Preserve replicas across worker recovery
  • ▸ OCR: stop task="parse" rejecting ordinary multi-page PDFs
  • ▸ vLLM: keep WorkerActor importable without vllm for multi-GPU launch
  • ▸ Stop polling requests flooding the access log
  • ▸ Declare audioread for qwen_omni_utils virtualenvs
  • ▸ Make cancelling a model launch actually cancel it
  • ▸ Normalize replica during autostart recovery
  • ▸ Repair pypiserver release dependencies
  • ▸ Mirror sdists with broken metadata builds

📚 Documentation

  • ▸ Update README models and links for v3.1.0
  • ▸ Add v3.1.0 to the release notes index
  • ▸ Fix broken star history chart links
  • ▸ Update README new model highlights