Skip to content

OpenAI API Compatibility

CompactifAI API provides OpenAI-compatible endpoints that allow you to use existing applications and libraries that work with OpenAI’s API with minimal changes. This compatibility layer makes it easy to quickly evaluate CompactifAI models, switching from other providers with just a few code changes.

  1. Replace OpenAI API key with your CompactifAI key
  2. Update base URL to your CompactifAI endpoint
  3. Use CompactifAI model names (e.g., “carina-60b”)
from openai import OpenAI
# Initialize the client with your CompactifAI API endpoint
client = OpenAI(
api_key="your-compactifai-api-key", # CompactifAI API key
base_url="https://your-compactifai-api-endpoint/v1" # Replace with your endpoint
)
# Chat completions
chat_completion = client.chat.completions.create(
model="carina-60b", # Use any one of the available CompactifAI model
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello, what can you tell me about CompactifAI?"}
],
temperature=0.7,
max_tokens=256
)
print(chat_completion.choices[0].message.content)
# Streaming chat completions
stream = client.chat.completions.create(
model="carina-60b",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me about artificial intelligence"}
],
temperature=0.7,
max_tokens=256,
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")
# Audio transcriptions (JSON only)
with open("meeting_minutes.mp3", "rb") as audio_file:
transcription = client.audio.transcriptions.create(
model="whisper-large-v3",
file=audio_file,
language="en",
temperature=0
)
print(transcription.text)
Endpoint Description
/v1/chat/completions Chat conversations
/v1/completions Text completions
/v1/responses Responses API (models with supports_responses: true only)
/v1/audio/transcriptions Speech-to-text transcription (non-streaming JSON)
/v1/models List available models
{
"model": "carina-60b",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello, I'm Camillo"},
{"role": "user", "content": "What is my name? What is the capital of Colombia?"}
],
"temperature": 0.7,
"max_tokens": 128,
"stop": ["###"],
"n": 1,
"user": "user-123"
}
{
"model": "carina-60b",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me about artificial intelligence"}
],
"temperature": 0.7,
"max_tokens": 128,
"stream": true,
"user": "user-123"
}
{
"model": "qwen-3-8-27b",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://cdn.britannica.com/86/170586-120-7E23E561/Taj-Mahal-Agra-India.jpg"
}
}
]
}
],
"temperature": 0.7,
"max_tokens": 128,
"stream": true,
"user": "user-123"
}

Only models that advertise supports_audio: true in GET /models accept audio in chat via an input_audio content part. No current model has supports_audio enabled, so this request shape is accepted only once a chat-audio-capable model is published. supports_audio covers chat only: transcription through /v1/audio/transcriptions is advertised separately as supports_audio_transcription.

{
"model": "MODEL_WITH_SUPPORTS_AUDIO_TRUE",
"messages": [
{
"role": "user",
"content": [
{ "type": "text", "text": "What is this audio file about?" },
{
"type": "input_audio",
"input_audio": { "data": "BASE64_AUDIO_HERE", "format": "wav" }
}
]
}
],
"max_tokens": 256
}

For models that advertise supports_video: true in GET /models, you can send a video in chat using a standard file content part.

{
"model": "qwen-3-8-27b",
"messages": [
{
"role": "user",
"content": [
{ "type": "text", "text": "Describe what happens in this video." },
{
"type": "file",
"file": {
"file_data": "data:video/mp4;base64,BASE64_VIDEO_HERE",
"filename": "clip.mp4"
}
}
]
}
]
}
{
"model": "carina-60b",
"prompt": "What is the capital of France?",
"temperature": 0.7,
"max_tokens": 128,
"stop": ["###"],
"user": "user-123"
}
{
"model": "carina-60b",
"input": "Say hello in one short sentence.",
"store": false,
"temperature": 0.7,
"max_output_tokens": 128,
"text": {
"format": {
"type": "text"
}
},
"reasoning": {
"effort": null,
"summary": null
}
}
{
"model": "carina-60b",
"input": "Tell me about artificial intelligence.",
"temperature": 0.7,
"max_output_tokens": 256,
"stream": true
}
from openai import OpenAI
client = OpenAI(
api_key="your-compactifai-api-key",
base_url="https://your-compactifai-api-endpoint/v1"
)
with open("meeting_minutes.mp3", "rb") as audio_file:
transcription = client.audio.transcriptions.create(
model="whisper-large-v3",
file=audio_file,
language="en",
temperature=0
)
print(transcription.text)

Responses are structured to be compatible with OpenAI’s format:

{
"id": "6a172d30ce8e4f34b4b830f8347c3911",
"created":1749600000,
"model": "carina-60b",
"object": "chat.completion",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello Camillo. It's nice to meet you.\n\nYour name is Camillo.\n\nThe capital of Colombia is Bogotá."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 25,
"completion_tokens": 35,
"total_tokens": 60
}
}
{
"id": "1861edc39ce648e3862a0b6ae9b7687b",
"object": "text_completion",
"created":1749600000,
"model": "carina-60b",
"choices": [
{
"text": "The capital of France is Paris.",
"index": 0,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 5,
"completion_tokens": 7,
"total_tokens": 12
}
}
{
"id": "resp_123",
"object": "response",
"created_at": 1749600000,
"model": "carina-60b",
"status": "completed",
"output": [
{
"type": "message",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "Hello! How can I help you today?"
}
]
}
],
"usage": {
"input_tokens": 12,
"output_tokens": 10,
"total_tokens": 22
},
"completed_at": 1749600005
}

The CompactifAI OpenAI compatibility layer supports the following endpoints:

Field Support Status
model Full (use CompactifAI model names)
messages Full (system, user, assistant roles)
max_tokens Full (legacy; deprecated in favor of max_completion_tokens)
min_tokens Full (CompactifAI extension)
temperature Full
top_p Full
n Partial (must be 1)
stream Full
stop Full
user Full
presence_penalty Full
frequency_penalty Full
logit_bias Ignored
logprobs Ignored
top_logprobs Ignored
seed Ignored
tools Full
tool_choice Full (required, auto, none, or a specific tool function)
function_call Ignored (deprecated)
function Ignored (deprecated)
parallel_tool_calls Ignored
response_format Full
max_completion_tokens Full (preferred over max_tokens)
data_sources Ignored (Azure-specific)
Field Support Status
id Full
created Full
model Full
object Full
choices Full
usage Full

For more details, see our API reference or have a look at the OpenAI chat completions documentation.

Field Support Status
model Full (use CompactifAI model names)
prompt Full
max_tokens Full
min_tokens Full (CompactifAI extension)
temperature Full
top_p Full
stop Full
user Full
best_of Ignored
echo Ignored
logit_bias Ignored
logprobs Ignored
seed Ignored
suffix Ignored
Field Support Status
id Full
created Full
model Full
object Full
choices Full
usage Full

For more details, see our API reference or have a look at the OpenAI text completions documentation.

Field Support Status
model Partial (must be a configured model with supports_responses: true in GET /v1/models)
input Full (required)
store Full (default true, same as OpenAI; send false to opt out)
file_id Full (optional; gateway resolves the file from the configured file store and attaches it to input as an input_text part: text files are decoded and PDFs have their text extracted)
input[].content[].type=input_file Partial (file_data, a data: file_url or file_id; the gateway reads the file and sends the model its text. PDF, Word (.docx), PowerPoint (.pptx), Excel (.xlsx; first 1,000 rows per sheet) and text files (txt, md, csv, tsv, json, xml, yaml) become input_text; images (png, jpeg, gif, webp) become input_image. Limits follow OpenAI: 50 MiB per file and 50 MiB per request. Differences from OpenAI: a remote http(s) file_url is not supported (400 unsupported_file_url); PDF page images are not sent, so scanned PDFs without a text layer are rejected; legacy .doc/.xls/.ppt are rejected. Safeguards, all configurable: at most 100 PDF pages, 500,000 characters of file text and 20 files per request, and 4 concurrent extractions per process (a 503 file_processing_busy when the server is busy))
background Partial (accepted; the response is always generated synchronously and returned in the same call)
prompt Partial (accepted and ignored; hosted prompt templates are not resolved, so id, version and variables have no effect)
file_id Full (optional; gateway resolves the file from the configured file store and attaches it to input — as an input_text part for text-decodable files, otherwise an input_file part)
temperature Full
max_output_tokens Full
stream Full
include Partial (code_interpreter_call.outputs, computer_call_output.output.image_url, file_search_call.results, message.input_image.image_url, message.output_text.logprobs, reasoning.encrypted_content are supported; web_search_call.action.sources and web_search_call.results run a web search when the request has a web_search tool; any other value, or an include that is not an array, is ignored)
Field Support Status
id Full
object Full
created_at Full
model Full
status Full
output Full
usage Full
completed_at Full
  • Authentication: Uses CompactifAI’s authentication
  • Models: Different model names (e.g., “carina-60b” instead of “gpt-4”)
  • Endpoint fields: Some OpenAI-specific fields not supported
  • Streaming usage: the final chunk always carries the request’s usage; set stream_options: {"include_usage": true} to also receive the running usage on every content chunk.
  • Responses: POST /v1/responses is supported for eligible models.

Tested with:

For other SDKs and libraries that are built to work with OpenAI’s API, you should be able to use them by changing the base URL to point to your CompactifAI API endpoint.

Python
from openai import OpenAI
client = OpenAI(
api_key='your-compactifai-api-key',
base_url='https://your-compactifai-api-endpoint/v1',
)
def main():
# Regular completion
completion = client.chat.completions.create(
model='carina-60b',
messages=[
{'role': 'system', 'content': 'You are a helpful assistant.'},
{'role': 'user', 'content': 'Tell me about CompactifAI.'}
],
temperature=0.7,
max_tokens=256
)
print(completion.choices[0].message.content)
# Streaming completion
stream = client.chat.completions.create(
model='carina-60b',
messages=[
{'role': 'system', 'content': 'You are a helpful assistant.'},
{'role': 'user', 'content': 'Tell me about artificial intelligence.'}
],
temperature=0.7,
max_tokens=256,
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")
if __name__ == "__main__":
main()

CompactifAI API maintains consistent error formats with the OpenAI API. However, the detailed error messages may differ. We recommend using the error messages primarily for logging and debugging purposes.

Errors are the OpenAI error object — message, type, param, code — with type always one of the five OpenAI categories. Success responses are the OpenAI chat.completion object, and streamed chunks are chat.completion.chunk, on every path, including a request blocked by guardrails. See Error handling for the codes.