Text Generation

CompactifAI API offers powerful text generation capabilities through our completion endpoints. Our compressed models can generate coherent, contextually relevant text for a wide range of applications while offering significant cost and performance advantages over uncompressed alternatives.

Basic Usage

import requests

API_URL = "https://api.compactif.ai/v1/completions"
API_KEY = "your_api_key_here"

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

data = {
    "model": "carina-60b",
    "prompt": "Write a short story about a robot that learns to paint:",
    "max_tokens": 250,
    "temperature": 0.7
}

response = requests.post(API_URL, headers=headers, json=data)
print(response.json()["choices"][0]["text"])

Parameters

Parameter	Type	Description
model	string	ID of the compressed model to use
prompt	string	The prompt to generate completions for
max_tokens	integer	Maximum number of tokens to generate
min_tokens	integer	Minimum number of tokens to generate
temperature	number	Controls randomness (0-2)
top_p	number	Controls diversity via nucleus sampling
n	integer	Number of completions to generate
stop	string or array	Sequences where the API will stop generating
tools	array	List of tools the model may call during generation
tool_choice	string	Controls tool usage ("auto", "none","required", or specific function)

Please refer to our API Reference or the OpenAI API reference for more information on the parameters.