Clarifai provides an easy-to-use platform to serve AI/ML models in production.

This guide walks you through deploying models on Clarifai — from scaffolding a project to running predictions in production.

Three commands to go from zero to a deployed model:

pip install -U clarifai

# 1. Authenticate
clarifai login

# 2. Scaffold a model (auto-selects GPU based on model size)
clarifai model init --toolkit vllm --model-name Qwen/Qwen3-0.6B

# 3. Deploy to production (auto-creates all infrastructure)
clarifai model deploy ./Qwen3-0.6B

The CLI handles compute cluster creation, nodepool setup, image building, and deployment automatically. When it finishes you'll see:

── Ready ──────────────────────────────────────────────
  Model deployed successfully!

  Model:       https://clarifai.com/you/main/models/qwen3-0-6b
  Deployment:  deploy-qwen3-0-6b-abc123
  Instance:    g5.xlarge

── Next Steps ─────────────────────────────────────────
  Predict:     clarifai model predict you/main/models/qwen3-0-6b "Hello"
  Logs:        clarifai model logs --deployment "deploy-qwen3-0-6b-abc123"
  Undeploy:    clarifai model undeploy --deployment "deploy-qwen3-0-6b-abc123"

• Quick Start
• Installation & Authentication
• Model Development Workflow
  • Step 1: Scaffold a Model
  • Step 2: Test Locally
  • Step 3: Deploy to Production
  • Step 4: Run Predictions
  • Step 5: Manage Your Deployment
• Writing a Custom Model
  • Model Folder Structure
  • config.yaml
  • model.py
  • requirements.txt
• Model Types
  • Python Model (ModelClass)
  • LLM / OpenAI-Compatible Model (OpenAIModelClass)
  • MCP Tool Server (MCPModelClass)
• Using the Python SDK Client
• Available Examples
• Supported Data Types
• Advanced Topics

pip install -U clarifai

# Interactive (prompts for your PAT)
clarifai login

# Non-interactive
clarifai login --pat $MY_PAT

# Org account
clarifai login --pat $PAT --user-id my-org

# Dev environment
clarifai login https://api-dev.clarifai.com

The CLI saves your credentials locally. Verify with:

clarifai whoami

You can manage multiple environments (prod, staging, org accounts) using named contexts:

clarifai config ls                  # List all contexts
clarifai config use <name>          # Switch active context

Note: You can generate a PAT in your Clarifai account under Personal Settings → Security.

init  →  serve (optional)  →  deploy  →  predict  →  manage
scaffold   test locally       push to      run         status / logs /
project    before deploying   production   inference   undeploy

Generate a ready-to-deploy model project with a single command:

# Deploy a HuggingFace LLM with vLLM
clarifai model init --toolkit vllm --model-name Qwen/Qwen3-0.6B

# Deploy with SGLang
clarifai model init --toolkit sglang --model-name Qwen/Qwen2-7B

# Deploy a HuggingFace model directly
clarifai model init --toolkit huggingface --model-name google/gemma-2b

# Wrap a local Ollama model
clarifai model init --toolkit ollama --model-name llama3.1

# Create an MCP tool server
clarifai model init --toolkit mcp my-mcp-server

# Wrap an OpenAI-compatible API
clarifai model init --toolkit openai my-wrapper

# Blank Python model (full control)
clarifai model init my-model

Available toolkits:

What it creates:

my-model/
├── config.yaml          # Simplified config (auto-filled at deploy time)
├── requirements.txt     # Dependencies
└── 1/
    └── model.py         # Model implementation

Smart GPU selection: For HuggingFace models, the CLI queries the model's metadata (parameter count, quantization, architecture) and auto-selects the smallest GPU instance that fits:

$ clarifai model init --toolkit vllm --model-name Qwen/Qwen3-4B
  Instance: g5.xlarge (Estimated 15.9 GiB VRAM, fits g5.xlarge (22 GiB))

Before deploying, test your model locally:

# Run in current Python environment (fastest)
clarifai model serve ./my-model

# Auto-create virtualenv and install deps
clarifai model serve ./my-model --mode env

# Build and run inside Docker (recommended for production parity)
clarifai model serve ./my-model --mode container

# Standalone gRPC server (no login required, offline development)
clarifai model serve ./my-model --grpc --port 9000

What happens (default API-connected mode):

1. Validates your config.yaml
2. Starts the model server locally
3. Registers with Clarifai so you can test via the Playground UI
4. Displays a predict command and Playground URL
5. Cleans up on Ctrl+C

Deploy your model to Clarifai cloud compute in one command. All infrastructure (compute cluster, nodepool, deployment) is created automatically:

# Deploy (uses instance from config.yaml)
clarifai model deploy ./my-model

# Override instance at deploy time
clarifai model deploy ./my-model --instance g6e.2xlarge

# GPU shorthands work too
clarifai model deploy ./my-model --instance a10g

# Specific cloud and region
clarifai model deploy ./my-model --instance g5.xlarge --cloud aws --region us-west-2

# Autoscaling
clarifai model deploy ./my-model --min-replicas 2 --max-replicas 10

# Deploy an already-uploaded model by URL
clarifai model deploy --model-url https://clarifai.com/user/app/models/id --instance g5.xlarge

Browse available instances:

$ clarifai list-instances

  Instance          Cloud   Region       GPUs  Accelerator  GPU Mem
  ──────────────────────────────────────────────────────────────────
  g5.xlarge         AWS     us-east-1    1     NVIDIA-A10G  24 GiB
  g6e.2xlarge       AWS     us-east-1    1     NVIDIA-L40S  48 GiB
  t3a.2xlarge       AWS     us-east-1    0     CPU          -
  n1-standard-4-t4  GCP     us-central1  1     NVIDIA-T4    16 GiB
  ...

Deploy phases:

── Validate ──  Config validation, HuggingFace repo access check
── Upload ────  Build and push model image
── Deploy ────  Create compute cluster, nodepool, deployment
── Monitor ───  Stream pod events until ready
── Ready ─────  Show model URL, predict command, status/log commands

Once deployed, run predictions directly from the CLI:

# Text input
clarifai model predict user/app/models/my-model "Hello world"

# Pipe from stdin
echo "Explain quantum computing" | clarifai model predict user/app/models/my-model

# Image/video/audio file
clarifai model predict user/app/models/detector --file photo.jpg

# Image URL
clarifai model predict user/app/models/detector --url https://example.com/img.jpg

# Named parameters
clarifai model predict user/app/models/llm -i prompt="Hello" -i max_tokens=200

# OpenAI chat mode
clarifai model predict openai/chat-completion/models/GPT-4 --chat "What is AI?"

# JSON output
clarifai model predict user/app/models/my-model "Hello" -o json

# Inspect available methods and their parameters
clarifai model predict user/app/models/my-model --info

Check status:

clarifai model status --deployment deploy-abc123
clarifai model status user/app/models/my-model

Stream logs:

# Model stdout/stderr
clarifai model logs --deployment deploy-abc123

# Kubernetes scheduling/scaling events
clarifai model logs --deployment deploy-abc123 --log-type events

# Print current logs and exit
clarifai model logs --deployment deploy-abc123 --no-follow

Remove a deployment:

clarifai model undeploy --deployment deploy-abc123

Most users don't need this section. If you're deploying a HuggingFace model, clarifai model init --toolkit vllm generates everything for you. Read on if you want to customize the generated code or build a model from scratch.

my-model/
├── config.yaml          # Model configuration
├── requirements.txt     # Python dependencies
└── 1/
    └── model.py         # Model implementation

The simplified config requires only a few fields. Everything else is auto-filled at deploy time:

model:
  id: "my-model"

build_info:
  python_version: "3.11"

compute:
  instance: g5.xlarge

checkpoints:
  repo_id: Qwen/Qwen3-0.6B
  type: huggingface
  when: runtime

Key fields:

Tip: user_id and app_id are resolved automatically from your CLI context — you don't need to put them in config.yaml.

Tip: For large models, use when: runtime to download checkpoints at runtime. This keeps Docker images small and speeds up builds.

Using a custom Docker base image (for vLLM, SGLang, etc.):

build_info:
  image: "lmsysorg/sglang:latest"

checkpoints:
  repo_id: Qwen/Qwen3-4B
  type: huggingface
  when: runtime

compute:
  instance: g5.xlarge

model:
  id: qwen3-4b

Your model must define a class that inherits from one of the base classes (ModelClass, OpenAIModelClass, or MCPModelClass) and expose at least one method decorated with @<BaseClass>.method.

from typing import Iterator
from clarifai.runners.models.model_class import ModelClass

class MyModel(ModelClass):

    def load_model(self):
        """Called once at startup. Load weights, initialize pipelines, etc."""
        self.model = ...

    @ModelClass.method
    def predict(self, prompt: str = "", max_tokens: int = 256) -> str:
        """Single request → single response."""
        return self.model(prompt, max_tokens=max_tokens)

    @ModelClass.method
    def generate(self, prompt: str = "") -> Iterator[str]:
        """Single request → streamed response."""
        for token in self.model.stream(prompt):
            yield token

Method types (determined by type hints):

load_model() (optional): Use this for expensive one-time initialization — loading model weights, creating pipelines, downloading checkpoints. Called once when the container starts.

Testing: When you run clarifai model serve, it loads your model and you can test via the Playground UI or CLI predict.

List all Python dependencies your model needs:

torch==2.5.1
transformers
accelerate

Tip: If your model requires PyTorch, specify the version explicitly (e.g., torch==2.5.1 or torch==2.4.1). Clarifai provides optimized pre-built images for these versions that significantly speed up builds and reduce cold start times.

Clarifai provides three base classes. Pick the one that matches your use case:

For custom inference logic — classification, detection, embeddings, or any Python code.

from clarifai.runners.models.model_class import ModelClass
from clarifai.runners.utils.data_types import Image, Text

class ImageCaptioner(ModelClass):

    def load_model(self):
        from transformers import pipeline
        self.pipe = pipeline("image-to-text", model="Salesforce/blip-image-captioning-base")

    @ModelClass.method
    def predict(self, image: Image) -> str:
        pil_image = image.to_pil()
        result = self.pipe(pil_image)
        return result[0]["generated_text"]

See: hello-world/ for a minimal example.

For models served via an OpenAI-compatible API (vLLM, SGLang, LMDeploy, etc.):

from typing import Iterator, List
from openai import OpenAI
from clarifai.runners.models.openai_class import OpenAIModelClass
from clarifai.runners.utils.data_utils import Param
from clarifai.runners.utils.openai_convertor import build_openai_messages

class MyLLM(OpenAIModelClass):
    client = OpenAI(api_key="local-key", base_url="http://localhost:8000/v1")
    model = client.models.list().data[0].id

    def load_model(self):
        """Start the inference server (vLLM, SGLang, etc.)."""
        pass

    @OpenAIModelClass.method
    def predict(
        self,
        prompt: str = "",
        chat_history: List[dict] = None,
        max_tokens: int = Param(default=256, description="Max tokens to generate."),
        temperature: float = Param(default=1.0, description="Sampling temperature."),
    ) -> str:
        messages = build_openai_messages(prompt, chat_history)
        response = self.client.chat.completions.create(
            model=self.model, messages=messages,
            max_completion_tokens=max_tokens, temperature=temperature,
        )
        return response.choices[0].message.content

    @OpenAIModelClass.method
    def generate(
        self,
        prompt: str = "",
        chat_history: List[dict] = None,
        max_tokens: int = Param(default=256, description="Max tokens to generate."),
        temperature: float = Param(default=1.0, description="Sampling temperature."),
    ) -> Iterator[str]:
        messages = build_openai_messages(prompt, chat_history)
        for chunk in self.client.chat.completions.create(
            model=self.model, messages=messages,
            max_completion_tokens=max_tokens, temperature=temperature, stream=True,
        ):
            if chunk.choices:
                yield chunk.choices[0].delta.content or ""

See: llm/vllm-qwen3-0.6B/ for a vLLM example, or llm/sglang-qwen3.5-4b/ for SGLang.

Tip: The fastest way to deploy an LLM is clarifai model init --toolkit vllm --model-name <hf-repo> — it generates the full model.py, config.yaml, and requirements.txt with the right server setup.

For deploying MCP tool servers that AI agents can use:

from fastmcp import FastMCP
from pydantic import Field
from clarifai.runners.models.mcp_class import MCPModelClass

server = FastMCP("my-tools", instructions="Useful tools for agents.", stateless_http=True)

@server.tool("hello", description="Say hello to someone")
def hello(name: str = Field(description="Name to greet")) -> str:
    return f"Hello, {name}!"

@server.resource("config://version")
def get_version():
    return "1.0.0"

class MyModel(MCPModelClass):
    def get_server(self) -> FastMCP:
        return server

See: mcp/math/ for a minimal example, or the mcp/ directory for real-world MCP servers (GitHub, Postgres, browser tools, code execution, etc.).

In addition to the CLI (clarifai model predict), you can call your deployed model from Python code. The SDK client mirrors the exact method signatures defined in your model.py:

from clarifai.client.model import Model

# Initialize by model URL
model = Model(model_url="https://clarifai.com/user/app/models/my-model")

# Or by IDs
model = Model(user_id="user", app_id="app", model_id="my-model")

Discover methods and signatures:

# List available methods
print(model.available_methods())

# Get method signature (parameter names, types, defaults)
print(model.method_signature(method_name="predict"))

# Generate a ready-to-use code snippet
print(model.generate_client_script())

Unary prediction (request → response):

result = model.predict(prompt="Hello world")
print(result)

# Batch prediction (multiple inputs)
results = model.predict([
    {"prompt": "Hello"},
    {"prompt": "World"},
])

Streaming prediction (request → stream of responses):

for chunk in model.generate(prompt="Explain quantum computing"):
    print(chunk, end="", flush=True)

Bidirectional streaming:

from clarifai.runners.utils.data_types import Audio

for transcript in model.transcribe(audio=iter(Audio(bytes=b"..."))):
    print(transcript.text)

Clarifai's model framework supports rich data typing for both inputs and outputs. Each method parameter and return type must be annotated.

Common types:

For the full reference including Concept, Region, Frame, NamedFields, and container types, see SUPPORTED_DATATYPE.md.

Models like Llama, Gemma, and other gated repos require an HuggingFace token:

# Set in environment (recommended)
export HF_TOKEN=hf_...

# Or add to config.yaml
checkpoints:
  repo_id: meta-llama/Llama-3.1-8B-Instruct
  hf_token: hf_...

The CLI validates HuggingFace access early (before building the Docker image) and provides actionable error messages if access is denied.

Note: When HF_TOKEN is in your environment, the CLI automatically persists it to config.yaml during deploy so the container can access it.

Browse all available GPU instances across cloud providers:

clarifai list-instances                      # All instances
clarifai list-instances --cloud aws          # AWS only
clarifai list-instances --gpu H100           # H100 instances
clarifai list-instances --min-gpus 2         # Multi-GPU
clarifai list-instances --min-gpu-mem 48Gi   # 48+ GiB GPU memory

If you want to upload a model image without immediately deploying it:

clarifai model upload ./my-model

You can deploy it later with:

clarifai model deploy --model-url https://clarifai.com/user/app/models/my-model --instance g5.xlarge

For classification or detection models that output concepts/labels, define them in config.yaml:

concepts:
  - id: "0"
    name: bus
  - id: "1"
    name: person
  - id: "2"
    name: car

Note: If your model loads from HuggingFace with checkpoints defined, concepts are inferred automatically.

Any command can use a different authentication context for a single invocation:

clarifai --context staging model deploy ./my-model
clarifai --context prod-openai model predict openai/chat-completion/models/GPT-4 "Hello"