🔐 Epic: Add Keycloak to Docker Compose - SSO-by-Default Development Environment

Goal

Add a pre-configured Keycloak instance to the docker-compose stack and enable SSO authentication by default, providing a fully functional SSO testing environment out of the box. Developers and QA can validate the complete OAuth2/OIDC login flow, role mapping, team assignment, and admin approval workflows without any external IdP setup.

Why Now?

MCP Gateway has comprehensive SSO support across 7 providers (GitHub, Google, IBM Security Verify, Microsoft Entra ID, Keycloak, Okta, Generic OIDC), but the docker-compose stack currently runs with email/password auth only. This creates several gaps:

1. No SSO Testing Path: Developers must manually set up an external IdP to test SSO flows — most skip it entirely
2. CI/CD Gap: Integration tests for SSO login, callback, user provisioning, and role mapping cannot run in docker-compose without a local IdP
3. Onboarding Friction: New contributors cannot experience or validate the SSO flow without provider credentials
4. Documentation Validation: SSO tutorials reference Keycloak extensively but there's no turnkey way to follow along
5. Role Mapping Testing: Keycloak role/group-to-RBAC mapping (_map_groups_to_roles, _sync_user_roles) needs a live IdP to exercise
6. Approval Workflow: The sso_require_admin_approval + pending approvals flow has no automated testing path
7. Load Testing Gap: The Locust load test suite (373/380 endpoints covered) cannot exercise SSO endpoints without a live provider

Keycloak is the ideal choice because it's open-source, self-hosted, supports auto-discovery (40% less configuration), and is already the most documented provider in the SSO tutorials.

📖 User Stories

Given I run `docker compose --profile sso up`
When Keycloak and Gateway are both healthy
Then the admin login page at http://localhost:8080/admin/login should show a "Keycloak" SSO button
And clicking it should redirect to the Keycloak login page
And logging in with test credentials (testuser@example.com / changeme) should complete the OAuth flow
And the user should be redirected back to the Gateway admin panel, authenticated
And a user record should be created in the Gateway database

Given Keycloak has the following pre-configured realm roles:
  - gateway-admin → maps to platform_admin
  - gateway-developer → maps to developer
  - gateway-viewer → maps to viewer
And test users are assigned these roles:
  - admin@example.com → gateway-admin (platform_admin in Gateway)
  - developer@example.com → gateway-developer (developer in Gateway)
  - viewer@example.com → gateway-viewer (viewer in Gateway)
When each user logs in via SSO
Then their Gateway RBAC roles should match the Keycloak role mapping
And SSO_KEYCLOAK_MAP_REALM_ROLES=true should be enabled
And role sync on login should work

Given the docker-compose.yml has a Keycloak service
When I run `docker compose up` (default, no profile)
Then Keycloak should NOT start (lightweight default)
And SSO should be disabled (SSO_ENABLED=false default)

When I run `docker compose --profile sso up`
Then Keycloak should start and become healthy
And the Gateway should have SSO_ENABLED=true
And the Keycloak SSO button should appear on the login page

When I run `docker compose --profile sso --profile monitoring up`
Then both Keycloak and the monitoring stack should be active

Given I run `docker compose --profile sso up`
When I access the Keycloak admin console at http://localhost:8180
Then I can log in with admin credentials (admin / changeme)
And I can see the pre-configured realm, client, users, and roles
And the tutorial steps (Step 2-8 in sso-keycloak-tutorial.md) match the pre-configured state
And the callback URL http://localhost:8080/auth/sso/callback/keycloak is pre-registered

Given SSO_REQUIRE_ADMIN_APPROVAL=true is set (optional config)
And a new user logs in via Keycloak SSO for the first time
When the OAuth callback completes
Then a pending approval record should be created
And the admin should see it at GET /auth/sso/pending-approvals
And the admin can approve/reject via POST /auth/sso/pending-approvals/{id}/action
And approved users should be created in the Gateway user database

Given the docker-compose testing profile includes Keycloak
When running Locust load tests with --profile sso --profile testing
Then the following SSO endpoints can be tested:
  - GET /auth/sso/providers (list providers)
  - GET /auth/sso/login/keycloak (initiate flow)
  - GET /auth/sso/callback/keycloak (callback - simulated)
  - POST /auth/sso/admin/providers (admin CRUD)
  - GET /auth/sso/admin/providers (admin list)
  - GET /auth/sso/pending-approvals (admin approvals)

🏗 Architecture

Docker Networking for OAuth

The key challenge is that OAuth requires URL consistency between browser redirects and server-to-server token exchange. In Docker Compose, the browser accesses services via localhost, while containers access each other via service names.

graph LR
    subgraph "Browser (Host)"
        B[Browser]
    end

    subgraph "Docker Network (mcpnet)"
        N[Nginx :80]
        G[Gateway :4444]
        K[Keycloak :8080]
    end

    B -->|"http://localhost:8080<br/>(admin UI, API, SSO callback)"| N
    B -->|"http://localhost:8180<br/>(Keycloak login page)"| K
    N -->|"http://gateway:4444"| G
    G -->|"http://keycloak:8080<br/>(token exchange, discovery)"| K
    K -->|"Redirect to<br/>http://localhost:8080/auth/sso/callback/keycloak"| B

Recommended Approach: Dual-URL with KC_HOSTNAME

sequenceDiagram
    participant B as Browser
    participant N as Nginx (localhost:8080)
    participant G as Gateway
    participant K as Keycloak (localhost:8180)

    B->>N: GET /admin/login
    N->>G: Proxy to gateway:4444
    G-->>B: Login page with "Keycloak" button

    B->>N: GET /auth/sso/login/keycloak
    N->>G: Proxy
    G->>K: GET /realms/mcp-gateway/.well-known/openid-configuration
    Note over G,K: Server-to-server via http://keycloak:8080
    K-->>G: Discovery JSON (URLs use http://localhost:8180)
    G-->>B: Redirect to http://localhost:8180/realms/mcp-gateway/protocol/openid-connect/auth

    B->>K: Keycloak login page
    B->>K: Submit credentials
    K-->>B: Redirect to http://localhost:8080/auth/sso/callback/keycloak?code=...

    B->>N: GET /auth/sso/callback/keycloak?code=...
    N->>G: Proxy
    G->>K: POST /realms/mcp-gateway/protocol/openid-connect/token
    Note over G,K: Token exchange via http://keycloak:8080
    K-->>G: Access token + ID token
    G->>K: GET /realms/mcp-gateway/protocol/openid-connect/userinfo
    K-->>G: User profile (email, name, roles, groups)
    G-->>B: Set JWT cookie + redirect to /admin

Key Networking Decision: Keycloak configured with KC_HOSTNAME=localhost and KC_HOSTNAME_PORT=8180 so OIDC discovery returns browser-accessible URLs. The gateway overrides the token/userinfo URLs to use internal http://keycloak:8080 for server-to-server communication.

Keycloak Realm Configuration

graph TB
    subgraph "Keycloak Realm: mcp-gateway"
        C[Client: mcp-gateway<br/>Confidential, Standard Flow]

        subgraph "Realm Roles"
            R1[gateway-admin]
            R2[gateway-developer]
            R3[gateway-viewer]
        end

        subgraph "Groups"
            G1[Administrators]
            G2[Developers]
            G3[Viewers]
        end

        subgraph "Test Users"
            U1["admin@example.com<br/>Role: gateway-admin<br/>Group: Administrators"]
            U2["developer@example.com<br/>Role: gateway-developer<br/>Group: Developers"]
            U3["viewer@example.com<br/>Role: gateway-viewer<br/>Group: Viewers"]
            U4["newuser@example.com<br/>No roles (tests default role)"]
        end

        subgraph "Client Scopes"
            S1[email]
            S2[profile]
            S3[roles]
            S4[groups - custom mapper]
        end
    end

    subgraph "Gateway RBAC Mapping"
        M1["gateway-admin → platform_admin"]
        M2["gateway-developer → developer"]
        M3["gateway-viewer → viewer"]
        M4["(no role) → viewer (default)"]
    end

    R1 --> M1
    R2 --> M2
    R3 --> M3

📋 Implementation Tasks

Phase 1: Keycloak Realm Configuration

• Create Keycloak realm export JSON

  • Create infra/keycloak/ directory
  • Create realm mcp-gateway with realm export JSON
  • Configure client mcp-gateway (confidential, standard flow)
  • Set valid redirect URIs: http://localhost:8080/auth/sso/callback/keycloak
  • Set web origins: http://localhost:8080
  • Create realm roles: gateway-admin, gateway-developer, gateway-viewer
  • Create groups: Administrators, Developers, Viewers
  • Add group membership mapper to include groups in tokens
  • Create test users with pre-assigned roles and groups
  • Set default client scopes: email, profile, roles
  • Configure user email verification as optional (dev mode)

• Create test user accounts

  • admin@example.com / changeme — role: gateway-admin, group: Administrators
  • developer@example.com / changeme — role: gateway-developer, group: Developers
  • viewer@example.com / changeme — role: gateway-viewer, group: Viewers
  • newuser@example.com / changeme — no role (tests default assignment)

Phase 2: Docker Compose Integration

• Add Keycloak service to docker-compose.yml

  • Use quay.io/keycloak/keycloak:latest image
  • Profile: sso (not started by default)
  • Expose port 8180:8080 (avoids conflict with nginx on 8080)
  • Set KC_HOSTNAME=localhost, KC_HOSTNAME_PORT=8180
  • Set KC_HTTP_ENABLED=true (dev mode, no TLS required)
  • Set admin credentials: KEYCLOAK_ADMIN=admin, KEYCLOAK_ADMIN_PASSWORD=changeme
  • Mount realm import: ./infra/keycloak/realm-export.json:/opt/keycloak/data/import/realm.json:ro
  • Command: start-dev --import-realm
  • Network: mcpnet
  • Health check: curl -f http://localhost:8080/health/ready (or exec-based)
  • Resource limits: 2 CPU, 2G memory
  • Add keycloakdata named volume for persistence

• Add SSO environment variables to gateway service

  • Add commented-out SSO block (disabled by default)
  • SSO_ENABLED=${SSO_ENABLED:-false} (overridden by profile)
  • SSO_KEYCLOAK_ENABLED=${SSO_KEYCLOAK_ENABLED:-false}
  • SSO_KEYCLOAK_BASE_URL=http://keycloak:8080
  • SSO_KEYCLOAK_REALM=mcp-gateway
  • SSO_KEYCLOAK_CLIENT_ID=mcp-gateway
  • SSO_KEYCLOAK_CLIENT_SECRET=<from-realm-export>
  • SSO_KEYCLOAK_MAP_REALM_ROLES=true
  • SSO_KEYCLOAK_MAP_CLIENT_ROLES=false
  • SSO_KEYCLOAK_GROUPS_CLAIM=groups
  • SSO_AUTO_CREATE_USERS=true
  • SSO_PRESERVE_ADMIN_AUTH=true

• Add SSO environment override file

  • Create docker-compose.sso.yml override file (alternative approach)
  • Or use environment variable substitution with SSO_ENABLED=${SSO_ENABLED:-false}
  • Ensure SSO activates only when --profile sso is used

• Add gateway dependency on keycloak

  • Conditional dependency when sso profile is active
  • Gateway should wait for Keycloak health before SSO bootstrap

Phase 3: Networking & URL Resolution

• Solve OAuth URL mismatch

  • Configure Keycloak KC_HOSTNAME=localhost, KC_HOSTNAME_PORT=8180
  • Verify OIDC discovery returns browser-accessible URLs
  • Verify gateway can reach Keycloak at http://keycloak:8080 for token exchange
  • Test that SSO_KEYCLOAK_BASE_URL=http://keycloak:8080 works for server-to-server
  • Verify authorization URL redirect works from browser at http://localhost:8180
  • Verify callback URL http://localhost:8080/auth/sso/callback/keycloak works through nginx

• Investigate nginx proxy option (alternative)

  • Evaluate proxying /realms/ through nginx to Keycloak
  • If viable, both browser and gateway use http://localhost:8080 (single URL)
  • Document trade-offs vs. dual-port approach

Phase 4: Makefile Targets

• Add SSO-related make targets
  • make compose-sso — Start stack with SSO profile
  • make compose-sso-monitoring — SSO + monitoring profiles
  • make compose-sso-testing — SSO + testing profiles (for Locust SSO testing)
  • make sso-test-login — Quick smoke test of SSO flow (curl-based)
  • Update make compose-up help text to mention SSO profile

Phase 5: Verification & Smoke Tests

• Create SSO smoke test script

  • Create scripts/test-sso-flow.sh or Python equivalent
  • Verify Keycloak is healthy and realm is imported
  • Verify /auth/sso/providers returns Keycloak
  • Verify /auth/sso/login/keycloak returns valid authorization URL
  • Simulate OAuth flow using Keycloak's direct access grant (API-based login)
  • Verify user is created in Gateway after SSO login
  • Verify role mapping is correct

• Integration test for SSO with Keycloak

  • Test user login → user creation → role assignment flow
  • Test role sync on subsequent login
  • Test with different user roles (admin, developer, viewer)
  • Test admin approval workflow (optional)

Phase 6: Documentation Updates

• Update Keycloak SSO tutorial

  • Add "Quick Start with Docker Compose" section at the top
  • Reference docker compose --profile sso up as the easiest way to get started
  • Note that the realm, client, and users are pre-configured

• Update docker-compose profile documentation

  • Add --profile sso to the profile list in docker-compose.yml header comments
  • Document Keycloak admin console access (http://localhost:8180, admin/changeme)
  • Document test user credentials
  • Add to deployment docs if applicable

• Update SSO overview page

  • Reference docker-compose setup as a quick-start option in docs/docs/manage/sso.md

Phase 7: Load Test Integration (Optional)

• Add SSO endpoint coverage to Locust
  • Add Locust user class for SSO admin endpoints
  • Test /auth/sso/providers, /auth/sso/admin/providers CRUD
  • Test /auth/sso/pending-approvals listing
  • Use Keycloak direct access grants to simulate token-based SSO login
  • Update coverage tracking in todo/locust-coverage.md

⚙️ Configuration Example

docker-compose.yml (new Keycloak service)

  # ──────────────────────────────────────────────────────────────────────
  # Keycloak - Open-source Identity Provider for SSO testing
  # Admin Console: http://localhost:8180 (admin / changeme)
  # Pre-configured realm: mcp-gateway (client, roles, test users)
  # Usage: docker compose --profile sso up -d
  # ──────────────────────────────────────────────────────────────────────
  keycloak:
    image: quay.io/keycloak/keycloak:latest
    restart: unless-stopped
    networks: [mcpnet]
    ports:
      - "8180:8080"    # Keycloak admin console + OIDC endpoints
    environment:
      - KEYCLOAK_ADMIN=admin
      - KEYCLOAK_ADMIN_PASSWORD=changeme
      # Hostname config: browser uses localhost:8180, containers use keycloak:8080
      - KC_HOSTNAME=localhost
      - KC_HOSTNAME_PORT=8180
      - KC_HTTP_ENABLED=true
      - KC_HEALTH_ENABLED=true
      - KC_METRICS_ENABLED=false
    command: ["start-dev", "--import-realm"]
    volumes:
      - ./infra/keycloak/realm-export.json:/opt/keycloak/data/import/realm.json:ro
      - keycloakdata:/opt/keycloak/data
    healthcheck:
      test: ["CMD-SHELL", "exec 3<>/dev/tcp/localhost/8080 && echo -e 'GET /health/ready HTTP/1.1\r\nHost: localhost\r\n\r\n' >&3 && cat <&3 | grep -q '\"status\":\"UP\"'"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 60s
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G
        reservations:
          cpus: '1'
          memory: 1G
    profiles: ["sso"]

Gateway SSO Environment Variables (added to gateway service)

      # ═══════════════════════════════════════════════════════════════════════════
      # SSO Configuration (enabled with --profile sso)
      # Requires Keycloak service. See: docs/docs/manage/sso-keycloak-tutorial.md
      # ═══════════════════════════════════════════════════════════════════════════
      - SSO_ENABLED=${SSO_ENABLED:-false}
      - SSO_KEYCLOAK_ENABLED=${SSO_KEYCLOAK_ENABLED:-false}
      - SSO_KEYCLOAK_BASE_URL=http://keycloak:8080
      - SSO_KEYCLOAK_REALM=mcp-gateway
      - SSO_KEYCLOAK_CLIENT_ID=mcp-gateway
      - SSO_KEYCLOAK_CLIENT_SECRET=${SSO_KEYCLOAK_CLIENT_SECRET:-keycloak-dev-secret}
      - SSO_KEYCLOAK_MAP_REALM_ROLES=true
      - SSO_KEYCLOAK_MAP_CLIENT_ROLES=false
      - SSO_KEYCLOAK_GROUPS_CLAIM=groups
      - SSO_AUTO_CREATE_USERS=true
      - SSO_PRESERVE_ADMIN_AUTH=true

Keycloak Realm Export (infra/keycloak/realm-export.json) — Summary

{
  "realm": "mcp-gateway",
  "enabled": true,
  "clients": [{
    "clientId": "mcp-gateway",
    "enabled": true,
    "clientAuthenticatorType": "client-secret",
    "secret": "keycloak-dev-secret",
    "redirectUris": ["http://localhost:8080/auth/sso/callback/keycloak"],
    "webOrigins": ["http://localhost:8080"],
    "standardFlowEnabled": true,
    "directAccessGrantsEnabled": true,
    "defaultClientScopes": ["email", "profile", "roles"]
  }],
  "roles": {
    "realm": [
      {"name": "gateway-admin", "description": "Maps to platform_admin"},
      {"name": "gateway-developer", "description": "Maps to developer"},
      {"name": "gateway-viewer", "description": "Maps to viewer"}
    ]
  },
  "groups": [
    {"name": "Administrators"},
    {"name": "Developers"},
    {"name": "Viewers"}
  ],
  "users": [
    {"username": "admin@example.com", "email": "admin@example.com", "enabled": true,
     "credentials": [{"type": "password", "value": "changeme"}],
     "realmRoles": ["gateway-admin"], "groups": ["Administrators"]},
    {"username": "developer@example.com", "email": "developer@example.com", "enabled": true,
     "credentials": [{"type": "password", "value": "changeme"}],
     "realmRoles": ["gateway-developer"], "groups": ["Developers"]},
    {"username": "viewer@example.com", "email": "viewer@example.com", "enabled": true,
     "credentials": [{"type": "password", "value": "changeme"}],
     "realmRoles": ["gateway-viewer"], "groups": ["Viewers"]},
    {"username": "newuser@example.com", "email": "newuser@example.com", "enabled": true,
     "credentials": [{"type": "password", "value": "changeme"}]}
  ]
}

✅ Success Criteria

• Zero-config SSO: docker compose --profile sso up provides a working SSO login flow
• Keycloak healthy: Service starts, imports realm, and passes health checks within 90s
• Login flow works: Browser can complete full OAuth2 authorization code flow with PKCE
• User provisioning: First-time SSO login creates a Gateway user with correct email and name
• Role mapping: Keycloak realm roles correctly map to Gateway RBAC roles
• Group mapping: Keycloak groups appear in JWT and are processed by Gateway
• Admin preserved: Local admin login (email/password) still works alongside SSO
• Profile isolation: Default docker compose up (no profile) does NOT start Keycloak
• Networking: OAuth URLs work correctly for both browser redirects and server-to-server exchange
• Documentation: Keycloak tutorial updated with docker-compose quick-start section
• Idempotent: Repeated docker compose down && docker compose --profile sso up works cleanly

🏁 Definition of Done

• infra/keycloak/realm-export.json created with realm, client, roles, groups, and test users
• Keycloak service added to docker-compose.yml under sso profile
• Gateway SSO environment variables configured (disabled by default, enabled via env override)
• OAuth networking validated: browser redirects + server-to-server token exchange both work
• Make targets added: make compose-sso
• SSO smoke test script validates the full login flow
• All 4 test users can log in and receive correct RBAC roles
• Keycloak admin console accessible at http://localhost:8180
• SSO tutorials updated with docker-compose quick-start
• Docker-compose profile header comments updated
• No changes to the default (non-SSO) startup path — backward compatible
• Code passes make verify checks

📝 Additional Notes

🔹 Docker Networking Challenge: OAuth requires the same URLs for browser redirects and API calls. Keycloak's KC_HOSTNAME / KC_HOSTNAME_PORT settings solve this by advertising external URLs while remaining accessible internally. The gateway uses http://keycloak:8080 for server-to-server communication (token exchange, discovery) while the browser accesses http://localhost:8180 for the login UI.

🔹 Profile vs. Default: Keycloak is placed behind the sso profile (not default) because:

• It adds ~1-2GB memory overhead
• Startup time increases by 30-60s
• Most development workflows don't need SSO
• Keeps the default stack lightweight

🔹 Realm Import Strategy: Keycloak's --import-realm flag imports the realm on first start only (skips if realm already exists). This is ideal for docker-compose where keycloakdata volume persists between restarts.

🔹 Test User Credentials: All test users use changeme as password for consistency with existing Gateway admin password defaults. This is acceptable for development only.

🔹 Future Enhancements:

• Add Keycloak to the tls profile with HTTPS configuration
• Add LDAP federation testing (Keycloak + OpenLDAP containers)
• Add multi-realm testing (dev + staging realms)
• Add Keycloak theme customization for branded login
• Integrate SSO endpoints into Locust load testing suite
• Add Playwright E2E tests for the browser-based SSO flow

🔹 Port Allocation:

🔗 Related Issues

• Keycloak SSO tutorial: docs/docs/manage/sso-keycloak-tutorial.md
• SSO overview: docs/docs/manage/sso.md
• Entra role mapping (reference implementation): docs/docs/manage/sso-entra-role-mapping.md
• SSO bootstrap: mcpgateway/utils/sso_bootstrap.py
• SSO service: mcpgateway/services/sso_service.py
• SSO config: mcpgateway/config.py (sso_keycloak_* variables)

📚 References

• Keycloak Docker Image
• Keycloak Realm Import/Export
• Keycloak Hostname Configuration
• OAuth2 in Docker Compose (common patterns)
• MCP Gateway SSO Architecture