A drop-in resilience layer for Python LLM applications. unsinkable routes any OpenAI-SDK-compatible client through TrueFoundry's AI Gateway so that provider outages, brownouts, and MCP tool failures fall back transparently.

• Live demo: https://web-demo-ebon-iota.vercel.app/
• Package: https://pypi.org/project/unsinkable/
• Source: https://github.com/0xNoramiya/unsinkable-ship

• Installation
• Quick Start
• Features
• Configuration
• Usage
  • Synchronous and asynchronous clients
  • Live dashboard
  • Chaos engineering
  • Resilient MCP client
  • Scripted demo
• TrueFoundry Setup
• Architecture
• Development
• Project Layout
• Acknowledgments
• License

pip install unsinkable                # core + OpenAI / Anthropic adapters
pip install "unsinkable[otel]"        # adds the OpenTelemetry exporter
pip install "unsinkable[codemod]"     # adds libcst for `unsinkable wire`

Requires Python 3.10+ and a TrueFoundry tenant with the AI Gateway enabled and at least one connected provider integration (OpenAI, Anthropic, Google Gemini, etc.).

from unsinkable import OpenAI

client = OpenAI()  # base_url, api_key, and observability injected from env

response = client.chat.completions.create(
    model="resilient-chat/resilient-chat",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

That is the complete change required to wrap an existing OpenAI-SDK call site. All other methods (embeddings, images, streaming, tool use, structured outputs, etc.) work unmodified.

Environment variables (a .env.example template is shipped in the repository):

Settings are loaded with pydantic-settings from .env or the process environment.

from unsinkable import OpenAI, AsyncOpenAI

sync_client = OpenAI()
async_client = AsyncOpenAI()

# Both expose the full openai-python surface area.
response = sync_client.chat.completions.create(
    model="resilient-chat/resilient-chat",
    messages=[{"role": "user", "content": "ping"}],
)

The shim is a subclass of openai.OpenAI / openai.AsyncOpenAI; any constructor argument supported by the upstream SDK is supported here. When http_client is provided explicitly, instrumentation is skipped and the caller takes full control of transport behavior.

unsinkable dashboard           # listens on http://127.0.0.1:8765 by default

Open the URL in a browser. The shim's instrumented transport posts every request to /events; the dashboard streams them to the page via Server-Sent Events and renders them in a live-updating table with provider badges, a latency sparkline, and stats counters.

The dashboard also exposes POST /api/chaos/{break,brownout,clear} endpoints and surfaces them as buttons in the UI, so demos can be driven entirely from the browser.

unsinkable chaos break openai          # gateway-side OpenAI fallback
unsinkable chaos break anthropic       # gateway-side Anthropic fallback
unsinkable chaos break cascade         # both providers down; gateway routes to Gemini
unsinkable chaos break mcp-primary     # primary MCP server skipped client-side
unsinkable chaos brownout 5            # adds 5 s of latency to every request
unsinkable chaos status                # show active scenario
unsinkable chaos clear                 # remove all active rules

Each scenario maps to a pre-created TrueFoundry Virtual Model whose priority-0 target is deliberately broken; the shim rewrites the outgoing model field so the gateway hits the broken target, fails for real, and falls back through its declared priority chain.

import asyncio
from unsinkable.mcp import ResilientMcpClient, McpBackend

backends = [
    McpBackend("primary",   "python", ["servers/primary.py"]),
    McpBackend("secondary", "python", ["servers/secondary.py"]),
]

async def main():
    async with ResilientMcpClient(backends) as mcp:
        result = await mcp.call_tool("web_search", {"query": "rust 1.80"})
    print(result)

asyncio.run(main())

ResilientMcpClient connects to each backend over stdio at context-manager entry. Tool calls are tried in declaration order; backends matching an active mcp-* chaos scenario are skipped, and exceptions from one backend trigger an automatic attempt on the next.

# Terminal 1
unsinkable dashboard

# Terminal 2
unsinkable demo

The demo command runs a 14-step scenario covering the happy path, a single provider outage with fallback to Claude, a brownout, a cascade outage with fallback to Gemini, MCP-layer failover, and recovery. Pair it with the dashboard for the full visual story.

The repository ships YAML manifests for the four Virtual Models referenced by the chaos scenarios. Setup takes roughly ten minutes.

1. Tenant + token. Sign in at https://<tenant>.truefoundry.cloud and create a Personal Access Token under Access → Personal Access Tokens. Copy .env.example to .env and populate TFY_API_KEY and TFY_HOST.

2. CLI installation and login.

   pip install -U truefoundry
   tfy login --host "$TFY_HOST" --api-key "$TFY_API_KEY"

3. Provider integrations. In the console, navigate to AI Gateway → Model Integrations → New and add the following five integrations:

   Integration name	Provider	Model	Notes
   openai	OpenAI	gpt-4o-mini	valid API key
   anthropic	Anthropic	claude-sonnet-4-6	valid API key
   google-gemini	Google AI Studio	gemini-2.5-flash-lite	valid API key
   openai-broken	OpenAI	gpt-4o	intentionally invalid key (e.g. sk-broken-on-purpose)
   anthropic-broken	Anthropic	claude-sonnet-4-6	intentionally invalid key

4. Virtual Models. Apply the four manifests:

   tfy apply \
     -f gateway-config/resilient_chat.yaml \
     -f gateway-config/chaos_openai_down.yaml \
     -f gateway-config/chaos_anthropic_down.yaml \
     -f gateway-config/chaos_cascade.yaml

5. Verify. Run unsinkable doctor for a table view, or python examples/smoke_test.py for a scripted end-to-end check that exercises a direct provider call, the happy-path Virtual Model, and a chaos-triggered fallback.

                ┌────────────────────────────────────────────────────────┐
                │                       Your code                        │
                │    from unsinkable import OpenAI, AsyncOpenAI          │
                └──────────────────────┬─────────────────────────────────┘
                                       │
            ┌──────────────────────────┴──────────────────────────┐
            ▼                                                     ▼
  ┌──────────────────────┐                          ┌──────────────────────┐
  │  unsinkable.OpenAI   │                          │ ResilientMcpClient   │
  │  • base_url, auth    │                          │  • priority backends │
  │  • httpx transport   │                          │  • chaos-aware       │
  │    instrumentation   │                          │  • per-call failover │
  │  • body rewriting    │                          └──────────┬───────────┘
  │    via chaos rules   │                                     │
  └──────────┬───────────┘                                     │
             │                                                 ▼
             │                                       ┌───────────────────┐
             ▼                                       │   MCP servers     │
  ┌──────────────────────┐                           │   (stdio)         │
  │ TrueFoundry          │                           └───────────────────┘
  │ AI Gateway           │
  │  • Virtual Models    │
  │  • Priority routing  │
  │  • Fallback codes    │
  └──────────┬───────────┘
             │
             ▼
  ┌────────────────────────────────────────┐
  │ OpenAI · Anthropic · Google Gemini …   │
  └────────────────────────────────────────┘

Request events emitted by the transport are also POSTed to the optional local dashboard for live observability.

git clone https://github.com/0xNoramiya/unsinkable-ship.git
cd unsinkable-ship
python -m venv .venv
.venv/bin/pip install -e ".[dev]"
.venv/bin/pytest -q

The test suite (11 tests) covers configuration loading, the chaos-state lifecycle, and live MCP failover against two locally spawned stdio servers.

unsinkable-ship/
├── src/unsinkable/             # Python package
│   ├── client.py               # OpenAI / AsyncOpenAI shim + httpx transport
│   ├── chaos.py                # State persistence and scenario activation
│   ├── mcp.py                  # ResilientMcpClient
│   ├── dashboard.py            # FastAPI + SSE dashboard
│   ├── auto_demo.py            # Scripted demo runner
│   ├── cli.py                  # Click entry point: doctor / dashboard / demo / chaos
│   ├── config.py               # pydantic-settings configuration
│   └── events.py               # RequestEvent dataclass and HTTP sink
├── examples/                   # Sample agent and MCP servers
├── gateway-config/             # TrueFoundry Virtual Model manifests
├── tests/                      # pytest suite (config + chaos + MCP)
└── web-demo/                   # Static client-side dashboard mirror (Vercel)

Built on TrueFoundry's AI Gateway. Adapters wrap the OpenAI Python SDK and the Anthropic Python SDK. Tool-side resilience uses the Model Context Protocol.

Released under the MIT License.