MCPLab by Inspectr

🧪 Test and evaluate MCP Servers with LLMs

Test how well LLM agents use your MCP tools, compare different models, and track quality over time with automated testing and detailed reports.

Get Started View on GitHub

localhost:5173

MCPLab dashboard showing evaluation results and test scenarios

See It in Action

Rich visual reports, detailed traces, and interactive dashboards.

Dashboard

Run Evaluation

Evaluation Results

Run Detail

MCP Analysis

Tool Analysis

Reference Reports

AI Assistant

Track pass rates, latency trends and recent runs at a glance.

Core Capabilities

• HTTP SSE Transport for MCP servers
• Multi-LLM support (OpenAI, Claude, Azure)
• Rich assertions & variance testing
• Detailed JSONL trace logs

Analysis & Reporting

• Trend analysis & LLM comparison
• HTML, JSON, Markdown outputs
• Custom metrics & KPI tracking
• Markdown reports for each run

Developer Experience

• CI-friendly CLI for scheduled runs
• Snapshot regression detection
• Interactive HTML reports
• Multi-agent testing via CLI

Quick Start

Up and running in under a minute.

1. Install

$ npx @inspectr/mcplab --help

2. Create eval config

servers:
  my-server:
    transport: "http"
    url: "http://localhost:3000/mcp"

agents:
  claude:
    provider: "anthropic"
    model: "claude-haiku-4-5-20251001"
    temperature: 0

scenarios:
  - id: "basic-test"
    agent: "claude"
    servers: ["my-server"]
    prompt: "Use the tools to complete this task..."
    eval:
      tool_constraints:
        required_tools: ["my_tool"]
      response_assertions:
        - type: "regex"
          pattern: "success|completed"

3. Run evaluation

$ npx @inspectr/mcplab run -c eval.yaml