Skip to content

Chat Completion

CompactifAI API’s chat completion endpoint enables you to create dynamic, multi-turn conversations with our advanced compressed language models, offering exceptional performance at significantly reduced costs.

import requests
API_URL = "https://api.compactif.ai/v1/chat/completions"
API_KEY = "your_api_key_here"
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json"
}
data = {
"model": "carina-60b",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me about quantum computing."}
],
"temperature": 0.7
}
response = requests.post(API_URL, headers=headers, json=data)
print(response.json()["choices"][0]["message"]["content"])
Field Type Description
model string ID of the compressed model to use
messages array Array of message objects
temperature number Controls randomness (0-2)
top_p number Controls diversity via nucleus sampling. 0 selects greedy sampling (deterministic output)
n integer Number of completions to generate
max_tokens integer Legacy. Maximum number of tokens to generate. Deprecated in favor of max_completion_tokens; kept for backwards compatibility. When both are sent, they are forwarded to the provider as-is.
max_completion_tokens integer Maximum number of tokens to generate in completion (preferred over max_tokens)
min_tokens integer Minimum number of tokens to generate
stream boolean Whether to stream back partial progress
stream_options object Streaming options. {"include_usage": true} returns the running usage on every chunk; when false (default) only the final chunk carries it.
stop string or array Sequences where the API will stop generating
tools array List of tools the model may call during generation. An empty array is the same as omitting tools.
tool_choice string Controls tool usage ("auto", "none","required", or specific function)
reasoning_effort string Reasoning effort for models that support it: none, minimal, low, medium, high, xhigh, or max. Deprecated in favour of reasoning.effort, which takes priority; any control that turns reasoning off (none, reasoning_enabled: false) wins over the others
chat_template_kwargs object Extra keyword arguments forwarded to the model’s chat template (advanced)

Add a web_search tool and set enable to true:

{
"model": "glm-5-3",
"messages": [{"role": "user", "content": "What is the latest stable Python release?"}],
"tools": [{"type": "web_search", "web_search": {"enable": true, "search_result": true}}]
}

The API searches the web on every request that carries an enabled web_search tool: it takes the last user message as the query (or search_query), puts the results in the prompt, and the model answers from them. The model does not decide whether to search on this endpoint.

web_search field Effect
enable Must be true; the default false ignores the tool.
search_query Search this query instead of the last user message.
count, content_size, search_domain_filter, search_recency_filter Passed to the search.
search_result Returns the results the model received in the response’s web_search array.
search_prompt Template that carries the results to the model; {{search_result}} and {{current_date}} are replaced.
result_sequence, require_search, search_engine Accepted and ignored; no effect.

The response carries a request_id. With stream: true the web_search sources arrive on the closing chunk (the one with finish_reason), and the stream carries no request_id. If the search backend fails the request fails with a 500 (code 1234); if web search is not enabled on the deployment it fails with a 400 (code 1221).

To get the search results without a model, call POST /v1/web_search.

Messages must be an array of objects with the following structure:

{
"role": "system" | "user" | "assistant",
"content": "The message content"
}

Please refer to our API Reference or the OpenAI API reference for more information on the fields.