Chat Completion
CompactifAI API’s chat completion endpoint enables you to create dynamic, multi-turn conversations with our advanced compressed language models, offering exceptional performance at significantly reduced costs.
Basic Usage
Section titled “Basic Usage”import requests
API_URL = "https://api.compactif.ai/v1/chat/completions"API_KEY = "your_api_key_here"
headers = { "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
data = { "model": "carina-60b", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Tell me about quantum computing."} ], "temperature": 0.7}
response = requests.post(API_URL, headers=headers, json=data)print(response.json()["choices"][0]["message"]["content"])Fields
Section titled “Fields”| Field | Type | Description |
|---|---|---|
model |
string | ID of the compressed model to use |
messages |
array | Array of message objects |
temperature |
number | Controls randomness (0-2) |
top_p |
number | Controls diversity via nucleus sampling. 0 selects greedy sampling (deterministic output) |
n |
integer | Number of completions to generate |
max_tokens |
integer | Legacy. Maximum number of tokens to generate. Deprecated in favor of max_completion_tokens; kept for backwards compatibility. When both are sent, they are forwarded to the provider as-is. |
max_completion_tokens |
integer | Maximum number of tokens to generate in completion (preferred over max_tokens) |
min_tokens |
integer | Minimum number of tokens to generate |
stream |
boolean | Whether to stream back partial progress |
stream_options |
object | Streaming options. {"include_usage": true} returns the running usage on every chunk; when false (default) only the final chunk carries it. |
stop |
string or array | Sequences where the API will stop generating |
tools |
array | List of tools the model may call during generation. An empty array is the same as omitting tools. |
tool_choice |
string | Controls tool usage ("auto", "none","required", or specific function) |
reasoning_effort |
string | Reasoning effort for models that support it: none, minimal, low, medium, high, xhigh, or max. Deprecated in favour of reasoning.effort, which takes priority; any control that turns reasoning off (none, reasoning_enabled: false) wins over the others |
chat_template_kwargs |
object | Extra keyword arguments forwarded to the model’s chat template (advanced) |
Web search
Section titled “Web search”Add a web_search tool and set enable to true:
{ "model": "glm-5-3", "messages": [{"role": "user", "content": "What is the latest stable Python release?"}], "tools": [{"type": "web_search", "web_search": {"enable": true, "search_result": true}}]}The API searches the web on every request that carries an enabled web_search tool: it takes the last user message as the query (or search_query), puts the results in the prompt, and the model answers from them. The model does not decide whether to search on this endpoint.
web_search field |
Effect |
|---|---|
enable |
Must be true; the default false ignores the tool. |
search_query |
Search this query instead of the last user message. |
count, content_size, search_domain_filter, search_recency_filter |
Passed to the search. |
search_result |
Returns the results the model received in the response’s web_search array. |
search_prompt |
Template that carries the results to the model; {{search_result}} and {{current_date}} are replaced. |
result_sequence, require_search, search_engine |
Accepted and ignored; no effect. |
The response carries a request_id. With stream: true the web_search sources arrive on the closing chunk (the one with finish_reason), and the stream carries no request_id. If the search backend fails the request fails with a 500 (code 1234); if web search is not enabled on the deployment it fails with a 400 (code 1221).
To get the search results without a model, call POST /v1/web_search.
Message Format
Section titled “Message Format”Messages must be an array of objects with the following structure:
{ "role": "system" | "user" | "assistant", "content": "The message content"}Please refer to our API Reference or the OpenAI API reference for more information on the fields.