What is an OpenAI-compatible API? What it covers and what to test

An OpenAI-compatible API is an API from another provider or server that accepts OpenAI-format requests for the endpoints it supports. It uses the same paths, JSON fields and bearer-key authentication. The official OpenAI SDK can call it once you change the base URL, key and model name. It is a format, not OpenAI.

Some engines offer both. vLLM also serves Anthropic-style and Cohere-style endpoints, and Ollama has its own native API.

What does an OpenAI-compatible API cover in practice?

It covers a set of endpoints, not the whole of OpenAI's platform. OpenAI's API reference shows the pattern: a base URL of https://api.openai.com/v1 and a bearer token in the Authorization header.

Compatible servers copy that pattern. As of October 2026, the vLLM documentation lists /v1/completions, /v1/chat/completions, /v1/embeddings, /v1/responses and /v1/audio/transcriptions.

Coverage is often partial. As of October 2026, the Ollama documentation says it supports "a subset of the OpenAI API". The vendors below document different gaps. vLLM ignores some parameters. Anthropic's layer does not support audio input or prompt caching. Gemini's does not support batch file uploads.

Can I use the OpenAI SDK with other models?

Yes. The official Python SDK accepts a base_url argument or an OPENAI_BASE_URL environment variable. You use the model name and key from the provider you are calling.

from openai import OpenAI

client = OpenAI(
    base_url="https://your-provider.example/v1",
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="your-model-name",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

The code change is small, but the testing takes longer. Our step-by-step guide, moving to a self-hosted OpenAI-compatible API, covers the switch itself.

Which providers and engines offer one?

Many providers and engines do. The table shows what each says in its own documentation, as of October 2026.

So yes, vLLM has an OpenAI-compatible server. The Anthropic API also has a compatibility layer, but Anthropic points to its native Claude API for the full feature set.

A native API can expose features the compatible layer lacks. As of October 2026, Anthropic's documentation says prompt caching is not supported in its compatibility layer but is supported in its own SDKs.

Does OpenAI-compatible mean the same results?

No. The label describes the request format. It says nothing about model quality, output style or how strictly a server follows each parameter.

As of October 2026, OpenAI says the Responses API is recommended for all new projects, while Chat Completions remains supported. Most of the providers above document Chat Completions, and vLLM and Ollama also list /v1/responses.

Before you trust the label, test the points below. The vendor details are from the documentation linked above, as of October 2026.

1. List the endpoints your code calls, then confirm each one exists. Chat, embeddings and the model list are separate checks.
2. Check for ignored parameters. A server can accept a parameter and do nothing with it. vLLM ignores user on chat completions, for example.
3. Send a streamed request and check that the chunks parse in your client.
4. Test tool calling and structured output. Anthropic says its layer ignores the strict parameter, so tool-use JSON is not guaranteed to follow your schema. Test your own schemas on any provider.
5. For embeddings, check the vector size and the model name before you re-index anything.
6. Trigger a bad key, an unknown model and a rate limit. Make sure your retry logic reads the errors correctly.
7. Run your own prompts on the new model. Different models answer differently, even behind the same API.

How Xinference supports OpenAI-compatible APIs

Xinference is an AI inference platform for enterprise teams. It runs in your own environment, whether that is your VPC, your data centre or your servers.

It puts one OpenAI-compatible API in front of every model. Moving from another OpenAI-compatible provider is a base URL and API key change, not a rewrite. You still pick the model name and run the checks above before you switch. Our developer page lists the Xinference endpoints.

If you want to run models in your own environment, see dedicated inference. The platform supports more than 300 individual models across language, embedding, reranking, speech, image and multimodal tasks, with models in the same family counted separately. Don't see the model you need? Talk to us about adding it.

If you would rather pay by the token, Model API serves open models behind the same API. Contact sales for pricing. For background on the layer underneath, read what is LLM inference.