Skip to content

[TESTING][FUNCTIONALITY]: Configuration manual test plan (env vars, runtime config, validation) #2434

Open
Task
Open
[TESTING][FUNCTIONALITY]: Configuration manual test plan (env vars, runtime config, validation)#2434
Task
Labels
SHOULDP2: Important but not vital; high-value items that are not crucial for the immediate releasechoreLinting, formatting, dependency hygiene, or project maintenance choresmanual-testingManual testing / test planning issuesreadyValidated, ready-to-work-on itemstestingTesting (unit, e2e, manual, automated, etc)
Milestone
Release 1.1.0
@crivetimihai

Description

@crivetimihai
crivetimihai
opened on Jan 25, 2026

⚙️ [TESTING][FUNCTIONALITY]: Configuration Manual Test Plan

Goal

Produce a comprehensive manual test plan for configuration functionality including environment variables, runtime configuration, validation, and defaults.

Why Now?

Configuration drives gateway behavior:

  1. Flexibility: Adapt to different environments
  2. Validation: Catch misconfiguration early
  3. Defaults: Sensible out-of-box experience
  4. Security: Protect sensitive values

📖 User Stories

US-1: Admin - Environment Configuration

As an administrator
I want to configure via environment variables
So that I can deploy in different environments

Acceptance Criteria:

Feature: Environment Configuration

  Scenario: Set configuration
    Given environment variables are set
    When the gateway starts
    Then it should use those values

  Scenario: Validation
    Given invalid configuration
    When the gateway starts
    Then it should fail with clear error

🏗 Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                      CONFIGURATION ARCHITECTURE                              │
└─────────────────────────────────────────────────────────────────────────────┘

    ENVIRONMENT             CONFIG                     APPLICATION
    ───────────             ──────                     ───────────

  ┌──────────────┐       ┌──────────────┐       ┌──────────────┐
  │ .env file    │       │  Pydantic    │       │  Gateway     │
  │ ENV vars     │──────▶│  Settings    │──────▶│  Application │
  │ CLI args     │       │              │       │              │
  └──────────────┘       └──────────────┘       └──────────────┘
                                │
                                │ Validation
                                ▼
                         ┌──────────────┐
                         │  Type check  │
                         │  Range check │
                         │  Defaults    │
                         └──────────────┘

📋 Test Environment Setup

export GATEWAY_URL="http://localhost:8000"

# Create .env file
cat > .env.test << 'EOF'
HOST=0.0.0.0
PORT=8000
JWT_SECRET_KEY=test-secret
DATABASE_URL=sqlite:///./test.db
EOF

🧪 Manual Test Cases

Section 1: Environment Variables

Case Scenario Variable Expected Validation
EV-01 Required var JWT_SECRET_KEY Used Value applied
EV-02 Missing required Unset Error Clear message
EV-03 Optional var LOG_LEVEL Default INFO used
EV-04 Override default LOG_LEVEL=DEBUG Overridden DEBUG used
EV-02: Missing Required Variable

Steps:

# Step 1: Start without required var
unset JWT_SECRET_KEY
make dev 2>&1 | head -20

# Expected: Error message about missing JWT_SECRET_KEY

Expected Result:

  • Startup fails
  • Clear error message about missing JWT_SECRET_KEY

Section 2: Validation

Case Scenario Value Expected Validation
VL-01 Valid URL postgresql://... Accepted Startup OK
VL-02 Invalid URL not-a-url Rejected Error
VL-03 Port range 65536 Rejected Error
VL-04 Type mismatch DEBUG=notbool Rejected Error
VL-02: Invalid URL Validation

Steps:

# Step 1: Set invalid database URL
DATABASE_URL="not-a-valid-url" make dev 2>&1 | head -20

# Expected: Validation error for DATABASE_URL

Expected Result:

  • Startup fails
  • Clear validation error

Section 3: Runtime Configuration

Case Scenario Method Expected Validation
RC-01 Get config GET API Config returned Values visible
RC-02 Update config PUT API Updated Change applied
RC-03 Sensitive hidden GET API Redacted Secrets hidden
RC-03: Sensitive Values Hidden

Steps:

# Step 1: Get configuration
curl -s "$GATEWAY_URL/api/admin/config" \
  -H "Authorization: Bearer $TOKEN" | jq .

# Step 2: Verify secrets are redacted
curl -s "$GATEWAY_URL/api/admin/config" \
  -H "Authorization: Bearer $TOKEN" | jq '.jwt_secret_key'
# Should show "***" or "[REDACTED]"

Expected Result:

  • Secrets are redacted in API response
  • Non-sensitive values shown

📊 Test Matrix

Test Case Env Vars Validation Runtime Defaults
EV-01 ✓
EV-02 ✓ ✓
EV-03 ✓ ✓
EV-04 ✓ ✓
VL-01 ✓
VL-02 ✓
VL-03 ✓
VL-04 ✓
RC-01 ✓
RC-02 ✓
RC-03 ✓

✅ Success Criteria

  • All 11 test cases pass
  • Required variables enforced
  • Invalid values rejected
  • Defaults applied correctly
  • Sensitive values protected

🔗 Related Files

  • mcpgateway/config.py
  • .env.example

🔗 Related Issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    SHOULDP2: Important but not vital; high-value items that are not crucial for the immediate releasechoreLinting, formatting, dependency hygiene, or project maintenance choresmanual-testingManual testing / test planning issuesreadyValidated, ready-to-work-on itemstestingTesting (unit, e2e, manual, automated, etc)

    Type

    Task

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions