Quickstart

Authentication

Every request should include your API key in the apikey HTTP header. Generate one on the Get API Key page, where you can also configure provider-specific keys that optionally override the server's default provider access.

# Using the OpenAI-compatible client (library accepts an api_key)
from openai import OpenAI
client = OpenAI(base_url="https://open-llm-2fgm.onrender.com/v1", api_key="YOUR_API_KEY")

# Or, using plain requests: include your key in the `apikey` header
import requests
resp = requests.get("https://open-llm-2fgm.onrender.com/v1/serverinfo", headers={"apikey": "YOUR_API_KEY"})
print(resp.json())

Making a request

Send a chat completion just like you would with OpenAI. Set model to any slug from the Models page.

response = client.chat.completions.create(
    model="model name",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain routing in one sentence."}
    ]
)

print(response.choices[0].message.content)

Request format

A chat completion accepts a model slug and an ordered array of messages. Optional fields like max_tokens let you cap the response length.

response = client.chat.completions.create(
    model="model name",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing simply."}
    ],
    max_tokens=500
)

• model — the slug of the model to route to, or "auto".
• messages — ordered chat turns with a role (system / user / assistant) and content.
• max_tokens — optional cap on the number of tokens generated.

Response format

Responses match OpenAI's standard chat completion shape. The generated text lives at choices[0].message.content, and token accounting is returned under usage.

{
  "id": "chatcmpl-abc123",
  "model": "groq...",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Quantum computing uses qubits that can be 0 and 1 at once..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 118,
    "total_tokens": 142
  }
}

Available models

Open LLM currently routes across three providers — Groq, Google Gemini and OpenRouter — spanning chat, vision and code models on free and paid tiers. Browse the full, always-up-to-date catalogue and copy any slug on the Models page.

Available providers

The server currently supports the following providers: Groq, OpenRouter, Google, HuggingFace, Opencode (Zen), Cerebras, and NVIDIA. Browse the full model list on the Models page.

Groq OpenRouter Google HuggingFace Opencode (Zen) Cerebras NVIDIA

Note: You can add your own provider-specific key on the Get API Key page using the POST /keys/provider endpoint. When set, that provider's requests will use your key.

Smart auto-routing

Not sure which model to pick? Set model="auto" and a lightweight AI agent evaluates each request — its length, complexity and intent — then picks the most suitable available model automatically, optimising for latency, cost and capability. It's ideal when you don't want to hand-select a model per request, and it applies automatic fallback if the chosen provider is unavailable.

# Let Open LLM choose the best model for you
response = client.chat.completions.create(
    model="openllm/auto",
    messages=[{"role": "user", "content": "Summarise this contract."}]
)

# The chosen model is returned in the response metadata
print(response.model)  # e.g. "gemini-pro"

Rate limits

Requests are rate-limited per API key. The current default is 20 requests/minute. If you exceed it, the API responds with a 429 Too Many Requests error — back off briefly and retry. Need a higher limit? Reach out and we'll bump your key.

Error handling

Errors use standard HTTP status codes. The most common ones:

Endpoints

This section summarizes the key endpoints, with exact request fields and examples using requests.

import requests

url = "https://open-llm-2fgm.onrender.com/v1"
headers = {"Content-Type": "application/json", "apikey": "YOUR_API_KEY"}
body = {
    "massage": "Write a short introduction to artificial intelligence in English.",
    "model": "gemini-pro"
}

resp = requests.post(url, json=body, headers=headers)
print(resp.status_code)
print(resp.json())

import requests
resp = requests.get("https://open-llm-2fgm.onrender.com/models")
print(resp.json())

import requests
resp = requests.get("https://open-llm-2fgm.onrender.com/model/gemini-pro")
print(resp.json())

import requests
headers = {"apikey": "YOUR_API_KEY"}
resp = requests.get("https://open-llm-2fgm.onrender.com/usage", headers=headers)
print(resp.json())

import requests
resp = requests.get("https://open-llm-2fgm.onrender.com/serverinfo")
print(resp.json())

import requests
headers = {"apikey": "YOUR_API_KEY"}
resp = requests.get("https://open-llm-2fgm.onrender.com/keys", headers=headers)
print(resp.json())

import requests
url = "https://open-llm-2fgm.onrender.com/keys/provider"
headers = {"Content-Type": "application/json", "apikey": "YOUR_API_KEY"}
body = {"provider": "huggingface", "provider_key": "hf_xxx..."}
resp = requests.post(url, json=body, headers=headers)
print(resp.json())

The exact behavior and field names for these endpoints are defined by the server implementation in server.py.