Enterprise
Sovereign AI, built for regulated enterprises.
Start on the Model API. When a workload needs it, move the same open models into your own cloud account or data centre, behind the same API.
Running production workloads for
XW Bank
From pilot to production
Your AI pilot worked. Now risk, finance and engineering have questions.
Where does our data go?
With Dedicated Inference, prompts and outputs are processed on infrastructure you control.
What will it cost at scale?
Open models cost a fraction of closed ones, and we price them below big-cloud rates.
What happens when the model changes?
One OpenAI-compatible API sits in front, so a new model is a config change, not a rewrite.
Open-model catalogue
LLMs, embeddings, speech and image models, plus your fine-tunes.
Engine choice
vLLM, SGLang, llama.cpp, Transformers or MLX.
GPU pooling
Shared across teams, scaled with traffic.
Deployment options
Start on the Model API. Move in-house when you are ready.
Every option runs the same API.
Model API
Pay per token for open models. The quickest way to start a pilot.
AWS Marketplace
Deploy into your AWS account and keep billing where it already is.
Your infrastructure
Behind your perimeter, including fully offline with no outbound connection.
Security and governance
Controls your risk team will ask about.
With Dedicated Inference, models run inside your own environment, so prompts and outputs are processed where you keep your data.
Inference in your environment
Processed on your hardware or in your cloud account.
No training on your data
We don’t use your prompts or outputs to train or fine-tune models unless you agree in writing.
Use cases by industry
What teams run on Xinference, by industry.
Where the model runs matters as much as how well it performs.
Financial services
Banks and insurers want AI on customer data, and their risk teams want to know exactly where that data is processed. Run open models inside your own environment and you can give them a straight answer.
Common workloads
KYC and onboarding
Pull details from ID documents and application forms, and flag what needs a human check.
Claims triage
Read claim notes and attachments, sort them by type and urgency, and send each one to the right team.
Contact centre
Summarise calls, draft replies and bring up the right policy while the customer is still on the line.
Document review
Check loan files, statements and contracts against your own rules and highlight what is missing.
How teams deploy it: Usually Dedicated Inference in your own cloud account or data centre. Some teams start on the Model API with non-customer data, then move.
In production at AIA and XW Bank.
Healthcare
Clinical text is some of the most sensitive data there is. Xinference installs inside your hospital network, including fully offline, so patient data stays on systems you run.
Common workloads
Clinical notes
Turn dictation and consult notes into structured summaries for a clinician to review.
Referral triage
Read incoming referrals, pull out the key details and sort them by urgency.
Clinical search
Search guidelines, policies and protocols in plain language.
Coding and admin
Suggest codes and fill in admin forms from the notes, for staff to confirm.
How teams deploy it: Dedicated Inference on your own servers, including fully offline with no outbound connection.
Government
Agencies want to use AI without locking into one vendor’s model or sending records outside their environment. Run open models on agency hardware and swap models without re-procuring.
Common workloads
Correspondence
Draft replies to ministerial and public correspondence for staff to edit and approve.
FOI and records
Find, sort and summarise documents for FOI requests and records management.
Consultation analysis
Group thousands of submissions by theme and pull out representative quotes.
Policy research
Search legislation, briefs and internal guidance, and summarise what applies.
How teams deploy it: Dedicated Inference on agency hardware or in your own cloud account.
Legal
Client files are confidential by default. Firms want help reading and drafting without sending matter documents to a public AI service.
Common workloads
Contract review
Compare contracts against your playbook and flag clauses that need a lawyer’s attention.
Discovery
Sort and summarise large document sets so the team starts with what matters.
Drafting
Produce first drafts of letters, clauses and memos from your own precedents.
Knowledge search
Search past matters and precedents in plain language.
How teams deploy it: Dedicated Inference in your own environment. Some firms test on the Model API with non-client documents first.
Education and research
Universities and research groups want to run many different open models, often their own fine-tunes, on shared GPUs.
Common workloads
Research computing
Serve open models to many research groups from one shared GPU pool.
Fine-tuned models
Deploy your own fine-tunes behind the same API as every other model.
Student and staff services
Answer common questions from your own policies and handbooks.
Document processing
Extract data from papers, forms and archives at scale.
How teams deploy it: Dedicated Inference on university infrastructure, with the Model API for quick experiments.
Enterprise questions
Where is our data processed?
Where you choose. With Dedicated Inference, models run in your data centre or cloud account, so prompts and outputs are processed there.
Can we run open models on our own servers, fully offline?
Yes. Dedicated Inference installs in your data centre or cloud account, including fully offline environments with no outbound network connection.
Can we start on the Model API and move later?
Yes. Start per token on the Model API, then move a workload to Dedicated Inference in your own environment. Both use the same OpenAI-compatible API, so the move is a base URL change.
How do we migrate from a cloud inference API?
Point your existing OpenAI SDK at a new base URL. The API is OpenAI-compatible, so application code stays the same.
Is our data used to train models?
No. We don’t use your prompts or outputs to train or fine-tune models unless you agree in writing.
Which models can we run?
Open LLMs, embedding, speech and image models, plus your own fine-tunes. Don’t see the model you need? Talk to us about adding it.
Last updated