-
Notifications
You must be signed in to change notification settings - Fork 615
[EPIC][TESTING]: Achieve 100% Locust Load Test Coverage for REST APIs #2636
Copy link
Copy link
Open
Labels
SHOULDP2: Important but not vital; high-value items that are not crucial for the immediate releaseP2: Important but not vital; high-value items that are not crucial for the immediate releasechoreLinting, formatting, dependency hygiene, or project maintenance choresLinting, formatting, dependency hygiene, or project maintenance choresepicLarge feature spanning multiple issuesLarge feature spanning multiple issuespythonPython / backend development (FastAPI)Python / backend development (FastAPI)test-automationAutomated testingAutomated testingtestingTesting (unit, e2e, manual, automated, etc)Testing (unit, e2e, manual, automated, etc)
Milestone
Description
Overview
Expand Locust load test coverage to achieve 100% coverage of all testable REST API endpoints. Current coverage is ~70% after PR #2635.
Related PR: #2635 (test(loadtest): expand Locust API coverage from 45% to 70%)
Current State
| Category | Endpoints | Coverage |
|---|---|---|
| Fully Covered (100%) | 36 | 9% |
| Well Covered (70-99%) | 47 | 12% |
| Partially Covered (50-69%) | 30 | 8% |
| Low Coverage (20-49%) | 45 | 11% |
| Intentionally Skipped | 241 | 60% |
Effective coverage: ~70% of testable REST endpoints
Remaining Gaps by Section
Phase 1: Complete CRUD Operations (Low Effort, High Value)
Servers (73% → 100%)
-
PUT /servers/{server_id}- Update server
Prompts (80% → 100%)
-
PUT /prompts/{prompt_id}- Update prompt
RBAC (77% → 100%)
-
PUT /rbac/roles/{role_id}- Update role -
POST /rbac/users/{user_email}/roles- Assign role to user -
DELETE /rbac/users/{user_email}/roles/{role_id}- Remove role from user
Tokens (70% → 100%)
-
PUT /tokens/{token_id}- Update token -
DELETE /tokens/admin/{token_id}- Admin: delete any token -
POST /tokens/teams/{team_id}- Create team token
Protocol (80% → 100%)
-
POST /protocol/sampling/createMessage- Create sampling message
Phase 2: A2A Agents (40% → 100%)
-
POST /a2a- Create agent -
PUT /a2a/{agent_id}- Update agent -
DELETE /a2a/{agent_id}- Delete agent -
POST /a2a/{agent_id}/state- Set agent state -
POST /a2a/{agent_id}/toggle- Toggle agent -
POST /a2a/{agent_name}/invoke- Invoke agent
Phase 3: Teams Advanced Operations (56% → 100%)
-
PUT /teams/{team_id}- Update team -
PUT /teams/{team_id}/members/{user_email}- Update member role -
DELETE /teams/{team_id}/members/{user_email}- Remove member -
POST /teams/{team_id}/invitations- Create invitation -
DELETE /teams/invitations/{invitation_id}- Cancel invitation -
POST /teams/invitations/{token}/accept- Accept invitation -
POST /teams/{team_id}/join- Request to join -
POST /teams/{team_id}/join-requests/{id}/approve- Approve request -
DELETE /teams/{team_id}/join-requests/{id}- Reject request -
DELETE /teams/{team_id}/leave- Leave team
Phase 4: Auth Admin Operations (42% → 100%)
-
POST /auth/email/admin/users- Admin: create user -
GET /auth/email/admin/users/{user_email}- Admin: get user -
PUT /auth/email/admin/users/{user_email}- Admin: update user -
DELETE /auth/email/admin/users/{user_email}- Admin: delete user -
POST /auth/email/change-password- Change password -
POST /auth/email/register- Register new user
Phase 5: Export/Import Operations (33% → 100%)
-
POST /export/selective- Selective export -
POST /import- Import configuration -
POST /import/cleanup- Cleanup import -
GET /import/status/{import_id}- Get import status
Phase 6: OAuth Operations (29% → 100%)
-
GET /oauth/authorize/{gateway_id}- Authorize gateway -
GET /oauth/callback- OAuth callback -
POST /oauth/fetch-tools/{gateway_id}- Fetch tools after OAuth -
DELETE /oauth/registered-clients/{client_id}- Delete OAuth client -
GET /oauth/registered-clients/{gateway_id}- Get gateway's OAuth client
Phase 7: Reverse Proxy (25% → 75%)
-
DELETE /reverse-proxy/sessions/{session_id}- Delete session -
POST /reverse-proxy/sessions/{session_id}/request- Send request -
- SSE (skip - streaming)GET /reverse-proxy/sse/{session_id}
Intentionally Excluded
| Group | Endpoints | Reason |
|---|---|---|
| LLM Config | 14 | Requires LLM providers configured |
| LLM Chat | 6 | Requires LLM providers configured |
| Admin UI | 200 | HTMX partials, low value for load testing |
| SSE/Streaming | ~5 | Not suitable for load testing |
| Gateway CRUD | 3 | Causes instability (external MCP server calls) |
Proposed Implementation
- Phase 1-2: Add
EntityCRUDCompletionUserandA2AAgentUserclasses - Phase 3-4: Add
TeamsAdvancedUserandAuthAdminUserclasses - Phase 5-6: Add
ExportImportUserandOAuthFlowUserclasses - Phase 7: Add
ReverseProxyUserclass
Each phase should be tested independently before merging.
Acceptance Criteria
- All Phase 1-7 endpoints have load test coverage
- No new test failures introduced
- Coverage tracking document updated (
todo/locust-coverage.md) -
make load-test-clipasses with < 1% failure rate - PR comment updated with final coverage statistics
Known Blockers
- [BUG][EXPORT]: Selective export fails with 'Server' object has no attribute 'is_active' #2606 — Selective export fails with
'Server' object has no attribute 'is_active'(blocks Phase 5POST /export/selective)
Note: This issue supersedes #2609, which tracked the same goal at an earlier coverage baseline. Blockers #2607 and #2608 (referenced in #2609) have been resolved.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
SHOULDP2: Important but not vital; high-value items that are not crucial for the immediate releaseP2: Important but not vital; high-value items that are not crucial for the immediate releasechoreLinting, formatting, dependency hygiene, or project maintenance choresLinting, formatting, dependency hygiene, or project maintenance choresepicLarge feature spanning multiple issuesLarge feature spanning multiple issuespythonPython / backend development (FastAPI)Python / backend development (FastAPI)test-automationAutomated testingAutomated testingtestingTesting (unit, e2e, manual, automated, etc)Testing (unit, e2e, manual, automated, etc)