-
Notifications
You must be signed in to change notification settings - Fork 615
[TESTING] Documentation Site Test Plan #2499
Copy link
Copy link
Labels
MUSTP1: Non-negotiable, critical requirements without which the product is non-functional or unsafeP1: Non-negotiable, critical requirements without which the product is non-functional or unsafedocumentationImprovements or additions to documentationImprovements or additions to documentationmanual-testingManual testing / test planning issuesManual testing / test planning issuestestingTesting (unit, e2e, manual, automated, etc)Testing (unit, e2e, manual, automated, etc)
Milestone
Description
[TESTING] Documentation Site Test Plan
Goal
Validate that the entire documentation site at https://ibm.github.io/mcp-context-forge/ renders correctly, all links work, code examples are accurate, and the user experience is consistent across all 213 documentation pages.
Why Now?
Documentation is the primary resource for users adopting the MCP Gateway. Testing ensures:
- All pages render without errors
- Navigation works correctly
- Code examples are copy-paste ready and accurate
- Links (internal and external) are not broken
- Images and diagrams display properly
- Search functionality works
- Mobile responsiveness is maintained
User Stories
US-1: User Navigates Documentation
User Story
As a new user
I want to navigate the documentation site easily
So that I can find the information I need to use the gateway
Acceptance Criteria
Feature: Documentation Navigation
Scenario: Navigate via sidebar
Given I am on the documentation homepage
When I click on a category in the sidebar
Then the category should expand
And I should see the subcategories/pages
Scenario: Navigate via search
Given I am on any documentation page
When I search for "authentication"
Then relevant pages should appear in search results
And clicking a result should navigate to that page
Scenario: Navigate via breadcrumbs
Given I am on a nested documentation page
When I click a breadcrumb link
Then I should navigate to that parent section
Scenario: Mobile navigation
Given I am on a mobile device
When I tap the menu icon
Then the navigation sidebar should appear
And I should be able to navigate to any pageTechnical Requirements
- Sidebar navigation with collapsible sections
- Search functionality (likely Algolia or mkdocs search)
- Breadcrumb navigation
- Responsive design for mobile
US-2: User Follows Code Examples
User Story
As a developer
I want to copy and run code examples from the documentation
So that I can quickly implement features
Acceptance Criteria
Feature: Code Examples
Scenario: Copy code example
Given a documentation page with code blocks
When I click the copy button on a code block
Then the code should be copied to clipboard
And a success indicator should appear
Scenario: Run bash example
Given a documentation page with a bash code example
When I copy and run the command
Then the command should execute successfully
And the expected output should match documentation
Scenario: Run Python example
Given a documentation page with Python code
When I copy and run the code
Then the code should execute without errors
And produce the expected resultTechnical Requirements
- Code blocks with syntax highlighting
- Copy-to-clipboard functionality
- Accurate and tested examples
- Version-specific notes where applicable
Architecture
┌─────────────────────────────────────────────────────────────────────────────┐
│ Documentation Site Architecture │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Source: │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ docs/docs/ 213 Markdown files │ │
│ ├───────────────────────────────────────────────────────────────────────┤ │
│ │ ├── api/ API documentation │ │
│ │ ├── architecture/ Architecture docs + 36 ADRs │ │
│ │ ├── best-practices/ Best practices guides │ │
│ │ ├── deployment/ 15+ deployment guides │ │
│ │ ├── development/ Developer guides │ │
│ │ ├── faq/ Frequently asked questions │ │
│ │ ├── manage/ Management & observability │ │
│ │ ├── operations/ Operations guides │ │
│ │ ├── overview/ Product overview │ │
│ │ ├── testing/ Testing guides │ │
│ │ ├── tutorials/ Step-by-step tutorials │ │
│ │ └── using/ Usage guides (servers, clients, agents) │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │
│ Build: │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ MkDocs │───▶│ Build │───▶│ GitHub Pages │ │
│ │ Material │ │ Process │ │ (Static Site) │ │
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
│ │
│ Site URL: https://ibm.github.io/mcp-context-forge/ │
│ │
│ Key Pages to Test: │
│ ├── Homepage (index.md) │
│ ├── Quick Start guides │
│ ├── API Reference │
│ ├── Deployment guides (Docker, K8s, Cloud) │
│ ├── ADR (Architecture Decision Records) │
│ └── Tutorials │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Test Environment Setup
Prerequisites
- Web browser (Chrome, Firefox, Safari)
- Network access to https://ibm.github.io/mcp-context-forge/
- Link checker tool (optional: linkchecker, broken-link-checker)
- Local development environment for testing code examples
Setup Commands
# Clone repo for local doc testing
cd /home/cmihai/github/mcp-context-forge
# Install mkdocs dependencies
pip install mkdocs-material mkdocs-awesome-pages-plugin
# Build docs locally
cd docs && mkdocs build
# Serve docs locally for testing
mkdocs serve --dev-addr 127.0.0.1:8001
# Check for broken links (install: npm install -g broken-link-checker)
blc https://ibm.github.io/mcp-context-forge/ -ro --exclude external
# Count documentation pages
find docs/docs -name "*.md" | wc -lManual Test Cases
TC-1: Homepage and Navigation
| Step | Action | Expected Result |
|---|---|---|
| 1 | Open https://ibm.github.io/mcp-context-forge/ | Homepage loads |
| 2 | Verify hero section | Title, description, buttons visible |
| 3 | Click "Get Started" | Navigates to quick start |
| 4 | Test sidebar navigation | All sections expandable |
| 5 | Test search functionality | Search returns relevant results |
| 6 | Test footer links | All links work |
Pages to Verify
Homepage sections to check:
- Hero section with logo and tagline
- Feature highlights
- Quick start links
- Navigation sidebar
- Search bar
- Footer with links
TC-2: Deployment Documentation
| Step | Action | Expected Result |
|---|---|---|
| 1 | Navigate to Deployment section | Section loads |
| 2 | Verify Docker Compose guide | Complete with examples |
| 3 | Verify Kubernetes guide | Helm charts documented |
| 4 | Verify cloud deployment guides | AWS, Azure, GCP, IBM |
| 5 | Test code examples | Commands are accurate |
| 6 | Check for broken links | All links work |
Deployment Pages
docs/docs/deployment/
├── index.md
├── local.md
├── container.md
├── compose.md
├── kubernetes.md
├── helm.md
├── minikube.md
├── openshift.md
├── argocd.md
├── aws.md
├── azure.md
├── google-cloud-run.md
├── ibm-code-engine.md
├── fly-io.md
├── proxy-auth.md
└── postgresql-schema-configuration.md
TC-3: API Documentation
| Step | Action | Expected Result |
|---|---|---|
| 1 | Navigate to API section | Section loads |
| 2 | Verify endpoint documentation | All endpoints covered |
| 3 | Check request/response examples | Valid JSON examples |
| 4 | Verify authentication docs | Auth methods documented |
| 5 | Test curl examples | Commands work |
TC-4: Architecture Decision Records (ADRs)
| Step | Action | Expected Result |
|---|---|---|
| 1 | Navigate to Architecture section | Section loads |
| 2 | List all ADRs | 36+ ADRs visible |
| 3 | Open sample ADR | Full content renders |
| 4 | Check ADR format | Context, Decision, Consequences |
| 5 | Verify diagrams | Images display correctly |
Sample ADRs to Check
Key ADRs to verify:
- ADR-006: Gateway Tool Rate Limiting
- ADR-014: Security Headers CORS Middleware
- ADR-017: Adopt orjson JSON Serialization
- ADR-021: Built-in Proxy vs Service Mesh
- ADR-026: Hiredis Redis Parser
- ADR-030: Metrics Cleanup Rollup
TC-5: Tutorials
| Step | Action | Expected Result |
|---|---|---|
| 1 | Navigate to Tutorials section | Section loads |
| 2 | Select a tutorial | Full tutorial loads |
| 3 | Follow step-by-step | Steps are clear |
| 4 | Run code examples | Code works |
| 5 | Verify screenshots | Images display |
TC-6: Using Section (Servers, Clients, Agents)
| Step | Action | Expected Result |
|---|---|---|
| 1 | Navigate to Using section | Section loads |
| 2 | Check Servers subsection | All server types covered |
| 3 | Check Clients subsection | Client integration documented |
| 4 | Check Agents subsection | A2A agents documented |
| 5 | Check Plugins subsection | Plugin development documented |
TC-7: Mobile Responsiveness
| Step | Action | Expected Result |
|---|---|---|
| 1 | Open site on mobile device | Page loads correctly |
| 2 | Test hamburger menu | Navigation opens |
| 3 | Navigate between pages | Smooth navigation |
| 4 | View code blocks | Horizontal scroll works |
| 5 | Test search on mobile | Search works |
TC-8: Link Validation
| Step | Action | Expected Result |
|---|---|---|
| 1 | Run link checker | Tool executes |
| 2 | Check internal links | No 404s |
| 3 | Check external links | Links resolve |
| 4 | Check anchor links | Anchors exist |
| 5 | Check image links | Images load |
Link Checker Commands
# Using broken-link-checker
npm install -g broken-link-checker
blc https://ibm.github.io/mcp-context-forge/ -ro
# Using linkchecker
pip install linkchecker
linkchecker https://ibm.github.io/mcp-context-forge/
# Using wget for basic check
wget --spider -r -nd -nv -l 3 https://ibm.github.io/mcp-context-forge/ 2>&1 | grep -B1 'broken'Test Matrix
| Section | Pages | Navigation | Code Examples | Links | Images | Status |
|---|---|---|---|---|---|---|
| Overview | 5+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Deployment | 15+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| API | 5+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Architecture | 40+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Using/Servers | 30+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Using/Clients | 10+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Using/Agents | 5+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Using/Plugins | 10+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Tutorials | 10+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Best Practices | 5+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| FAQ | 5+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Operations | 5+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Development | 5+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
| Testing | 5+ | 🔲 | 🔲 | 🔲 | 🔲 | 🔲 |
Success Criteria
- All 213 documentation pages render without errors
- Sidebar navigation works for all sections
- Search returns relevant results for key terms
- No broken internal links
- No broken external links (or documented as known issues)
- All images and diagrams display correctly
- Code blocks have syntax highlighting
- Copy-to-clipboard works on code blocks
- Mobile navigation works on all pages
- Page load times are acceptable (<3s)
- Code examples are accurate and runnable
Definition of Done
- All sections manually reviewed
- Link checker run with zero internal broken links
- Sample code examples tested and working
- Mobile responsiveness verified
- Search functionality tested with 10+ queries
- Issues logged for any problems found
Related Files
docs/- Documentation source directorydocs/mkdocs.yml- MkDocs configurationdocs/docs/- Markdown source files (213 files).github/workflows/- GitHub Pages deployment workflow
Related Issues
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
MUSTP1: Non-negotiable, critical requirements without which the product is non-functional or unsafeP1: Non-negotiable, critical requirements without which the product is non-functional or unsafedocumentationImprovements or additions to documentationImprovements or additions to documentationmanual-testingManual testing / test planning issuesManual testing / test planning issuestestingTesting (unit, e2e, manual, automated, etc)Testing (unit, e2e, manual, automated, etc)