Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.portkey.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

Portkey’s AI Gateway provides a unified interface to 1,600+ LLMs with enterprise features: observability, automatic retries, fallbacks, caching, and cost controls—all through a simple API.

​
Quick Start

Get started in 3 steps: 
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add provider in Model Catalog (e.g., @openai-prod)
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",  # @provider-slug/model-name
    messages=[{"role": "user", "content": "What is a fractal?"}]
)

print(response.choices[0].message.content)
Portkey also supports Anthropic’s native /messages endpoint. See Using with Anthropic SDK below.

​
Add Provider in Model Catalog

Before making requests, add a provider:
  1. Go to Model Catalog → Add Provider
  2. Select your provider (OpenAI, Anthropic, etc.)
  3. Choose existing credentials or enter your API key
  4. Name your provider (e.g., openai-prod)
Your provider slug will be @openai-prod (the name you chose with @ prefix).

Complete Model Catalog Guide →

Set up budgets, rate limits, and manage credentials

​
Switch Between Providers

Change the model string to use different providers—same code, different models: 
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

# OpenAI
response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Anthropic
response = portkey.chat.completions.create(
    model="@anthropic-prod/claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": "Hello!"}],
    max_tokens=250  # Required for Anthropic
)

# Mistral
response = portkey.chat.completions.create(
    model="@mistral-prod/mistral-large-latest",
    messages=[{"role": "user", "content": "Hello!"}]
)

​
Examples

​
Vision

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/800px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"}}
        ]
    }],
    max_tokens=300
)

print(response.choices[0].message.content)

​
Function Calling

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

tools = [{
        "type": "function",
        "function": {
        "name": "get_weather",
        "description": "Get current weather in a location",
          "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City and state, e.g. San Francisco, CA"}
            },
            "required": ["location"]
        }
        }
}]

response = portkey.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Boston?"}],
  tools=tools,
  tool_choice="auto"
)

print(response.choices[0].message.tool_calls)

​
Image Generation

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

image = portkey.images.generate(
    model="@openai-prod/dall-e-3",
    prompt="A serene mountain landscape at sunset",
    size="1024x1024"
)

print(image.data[0].url)

​
Embeddings

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

embeddings = portkey.embeddings.create(
    model="@openai-prod/text-embedding-3-small",
    input=["Hello world", "Goodbye world"]
)

print(embeddings.data[0].embedding[:5])  # First 5 dimensions

​
Audio Transcription

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

transcription = portkey.audio.transcriptions.create(
    model="@openai-prod/whisper-1",
    file=open("/path/to/audio.mp3", "rb")
)

print(transcription.text)

​
Using with OpenAI SDK

Use your existing OpenAI code with Portkey—just change 2 parameters: 
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# Change base_url and api_key
client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url=PORTKEY_GATEWAY_URL
)

# Use model with @provider-slug prefix
response = client.chat.completions.create(
    model="@openai-prod/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

​
Using with Anthropic SDK

Portkey fully supports Anthropic’s native /messages endpoint. Use the Anthropic SDK directly with Portkey: 
import anthropic

client = anthropic.Anthropic(
    api_key="dummy", # we will use portkey's provider slug
    default_headers={"x-portkey-api-key": "YOUR_PORTKEY_API_KEY"},
    base_url="https://api.portkey.ai"
)

message = client.messages.create(
    model="@anthropic-prod/claude-sonnet-4-5-20250929",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude!"}]
)

print(message.content[0].text)
All Portkey features (caching, observability, configs) work with the Anthropic SDK—just add the x-portkey-* headers.

Anthropic Integration Guide →

Learn about prompt caching, extended thinking, and more Anthropic features

​
Gateway Features

Add production features through configs: 
from portkey_ai import Portkey

# Attach a config for retries, caching, fallbacks
portkey = Portkey(
        api_key="PORTKEY_API_KEY",
    config="pc-your-config-id"  # Created in Portkey dashboard
)

Automatic Retries

Retry failed requests with exponential backoff

Fallbacks

Automatically switch to backup providers

Caching

Cache responses to reduce costs and latency

Load Balancing

Distribute requests across multiple providers

Gateway Configs Guide →

Learn how to create and use configs

​
Supported Integrations

Portkey integrates with the entire AI ecosystem:

LLM Providers

1,600+ models from OpenAI, Anthropic, Google, Mistral, Cohere, and 30+ providers

Agent Frameworks

LangChain, CrewAI, AutoGen, OpenAI Agents, Strands, and more

Libraries

LangChain, LlamaIndex, Vercel AI SDK, and popular frameworks

Guardrails

Aporia, Pillar, Patronus, and content safety providers

Vector Databases

Pinecone, Weaviate, and vector store integrations

MCP Servers

Model Context Protocol servers and tools

​
Next Steps

Observability

Track costs, latency, and usage

Prompt Library

Manage and version prompts

Guardrails

Add PII detection and content filtering

Model Catalog

Manage providers, budgets, and access
Last modified on April 20, 2026