Structured Output

CompactifAI API’s chat completion endpoint supports Structured Output (also known as JSON Mode or tool calling), allowing you to define the exact JSON format in which the model should respond.

Basic Usage

import requests
import pydantic
import json

API_URL = "https://api.compactif.ai/v1/chat/completions"
API_KEY = "your_api_key_here"


class NameCity(pydantic.BaseModel):
    name: str
    city: str

openai_schema = {
    "name": "NameCity",
    "schema": NameCity.model_json_schema(),
    "strict": True
}
headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

data = {
    "messages": [
        {
            "role": "user",
            "content": "My name is Camilo. What is my name ? What is the capital of Colombia? Respond in JSON"
        }
    ],
    "model": "cai-deepseek-r1-0528-slim",
    "response_format": {
        "type": "json_schema",
        "json_schema": openai_schema
    }
}



response = requests.post(API_URL, headers=headers, json=data)

# Validate that the output follows the correct JSON schema

final_content = NameCity.model_validate(json.loads(response.json()['choices'][0]['message']['content']))

print(final_content)

Fields

Field	Subfield	Type	Description
model		string	ID of the compressed model to use
messages		array	Array of message objects
response_format		object	Dictionary with subfield explained below
	type	string	Define the response type. Allowed values: text – return natural-language text. json_object – Return a valid JSON object json_schema – Return a json object respecting a specific json schema
	json_schema	string	Define the json schema. Can only be defined if type=json_schema

Response Example

{
    "id": "chatcmpl-38109928-1f9c-4cf2-b3af-9c2bb32f6114",
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null,
            "message": {
                "content": "{\n  \"name\": \"Camilo\",\n  \"city\": \"Bogota\"\n}",
                "refusal": null,
                "role": "assistant",
                "annotations": null,
                "audio": null,
                "function_call": null,
                "tool_calls": [],
                "reasoning_content": null
            },
            "stop_reason": null
        }
    ],
    "created": 1761229600,
    "model": "cai-deepseek-r1-0528-slim",
    "object": "chat.completion",
    "service_tier": null,
    "system_fingerprint": null,
    "usage": {
        "completion_tokens": 22,
        "prompt_tokens": 25,
        "total_tokens": 47,
        "completion_tokens_details": null,
        "prompt_tokens_details": null
    },
    "prompt_logprobs": null,
    "kv_transfer_params": null
}

Compatibility

Model Name	Model ID	Structured Output compatible?
DeepSeek R1 0528 Slim by CompactifAI	cai-deepseek-r1-0528-slim	Yes
Llama 4 Scout Slim by CompactifAI	cai-llama-4-scout-slim	Yes
Llama 4 Scout	llama-4-scout	Yes
Llama 3.3 70B Slim by CompactifAI	cai-llama-3-3-70b-slim	Yes
Llama 3.3 70B	llama-3-3-70b	Yes
Llama 3.1 8B Slim by CompactifAI	cai-llama-3-1-8b-slim	Yes
Llama 3.1 8B Slim Reasoning by CompactifAI	cai-llama-3-1-8b-slim-r	Yes
Llama 3.1 8B	llama-3-1-8b	Yes
Mistral Small 3.1 Slim by CompactifAI	cai-mistral-small-3-1-slim	Yes
Mistral Small 3.1	mistral-small-3-1	Yes
OpenAI GPT OSS 20B	gpt-oss-20b	Yes
OpenAI GPT OSS 120B	gpt-oss-120b	Yes

Recommendations

• In the prompt, ask the model explicitly to create a JSON output
• In the prompt, include the json_schema to be used. This will help the token generation process