Tool Calling

CompactifAI API’s chat completion endpoint supports function calling (tool calling), allowing you to integrate custom capabilities directly into the conversation flow for more dynamic and precise responses.

Basic Usage

import requests

API_URL = "https://api.compactif.ai/v1/chat/completions"
API_KEY = "your_api_key_here"

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

data = {
    "messages": [
        {
            "role": "user",
            "content": "What's the weather like in San Sebastián?"
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Respond only with a get_weather generated by a tool. Get the current weather in a given location.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "City and state, e.g., 'San Sebastián'"
                        },
                        "unit": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"]
                        }
                    },
                    "required": ["location", "unit"]
                }
            }
        }
    ],
    "tool_choice": "auto",
    "model": "cai-llama-4-scout-slim"
}

response = requests.post(API_URL, headers=headers, json=data)

print(response.json())

Fields

Field	Type	Description
model	string	ID of the compressed model to use
messages	array	Array of message objects
tools	array	Array of tool objects defining available functions
tool_choice	string	Which function is called: “none”, “auto”,“required”, or specific function

Response Example

{
    "id": "chatcmpl-a7550b9e-83a2-4686-9f07-7000712eafdd",
    "choices": [
        {
            "finish_reason": "tool_calls",
            "index": 0,
            "logprobs": null,
            "message": {
                "content": null,
                "refusal": null,
                "role": "assistant",
                "annotations": null,
                "audio": null,
                "function_call": null,
                "tool_calls": [
                    {
                        "id": "chatcmpl-tool-c57ce1b880b84f4a9b3ddd5856e99f39",
                        "function": {
                            "arguments":"{\"location\": \"San Sebasti\án\", \"unit\": \"celsius\"}",
                            "name": "get_weather"
                        },
                        "type": "function"
                    }
                ],
                "reasoning_content": null
            },
            "stop_reason": null
        }
    ],
    "created": 1755007250,
    "model": "cai-llama-4-scout-slim",
    "object": "chat.completion",
    "service_tier": null,
    "system_fingerprint": null,
    "usage": {
        "completion_tokens": 13,
        "prompt_tokens": 684,
        "total_tokens": 697,
        "completion_tokens_details": null,
        "prompt_tokens_details": null
    },
    "prompt_logprobs": null,
    "kv_transfer_params": null
}

Compatibility

Model Name	Model ID	Function/Tool Calling Compatible?
DeepSeek R1 0528 Slim by CompactifAI	cai-deepseek-r1-0528-slim	No
DeepSeek R1 0528	deepseek-r1-0528	No
Llama 4 Scout Slim by CompactifAI	cai-llama-4-scout-slim	Yes
Llama 4 Scout	llama-4-scout	Yes
Llama 3.3 70B Slim by CompactifAI	cai-llama-3-3-70b-slim	Yes
Llama 3.3 70B	llama-3-3-70b	Yes
Llama 3.1 8B Slim by CompactifAI	cai-llama-3-1-8b-slim	Yes
Llama 3.1 8B Slim Reasoning by CompactifAI	cai-llama-3-1-8b-slim-r	Yes
Llama 3.1 8B	llama-3-1-8b	Yes
Mistral Small 3.1 Slim by CompactifAI	cai-mistral-small-3-1-slim	No
Mistral Small 3.1	mistral-small-3-1	No
OpenAI GPT OSS 20B	gpt-oss-20b	No
OpenAI GPT OSS 120B	gpt-oss-120b	No

Recommendations

• tool_choice can be set as a custom function (e.g. "tool_choice": {"type": "function", "function": {"name": "get_weather"}})
• Tool calling for GPT and DeepSeek based models is a work in progress