Skip to content

zilbonn/OWASP-WSTG-Rag

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
RAG_runner
RAG_runner
 
 
raw_data
raw_data
 
 
.gitignore
.gitignore
 
 
CLAUDE.md
CLAUDE.md
 
 
README.md
README.md
 
 

Repository files navigation

OWASP WSTG RAG

A Retrieval-Augmented Generation (RAG) system that indexes the OWASP Web Security Testing Guide (WSTG) into a vector database, providing instant access to security testing methodologies via REST API and MCP (Model Context Protocol) for Claude Code integration.

Features

  • Complete WSTG Coverage - All 12 WSTG testing categories indexed and searchable
  • Semantic Search - Find relevant testing methodologies using natural language queries
  • MCP Integration - Direct integration with Claude Code for AI-assisted penetration testing
  • REST API - HTTP endpoints for programmatic access
  • WSTG ID Lookup - Retrieve complete test cases by WSTG identifier (e.g., WSTG-INPV-05)

WSTG Categories

Category WSTG ID Description
Information Gathering WSTG-INFO Fingerprinting, enumeration, mapping
Configuration WSTG-CONF Server/platform configuration testing
Identity Management WSTG-IDNT User registration, account provisioning
Authentication WSTG-ATHN Login, password policy, MFA testing
Authorization WSTG-ATHZ Privilege escalation, IDOR, access control
Session Management WSTG-SESS Session tokens, cookies, fixation
Input Validation WSTG-INPV SQLi, XSS, command injection, SSTI
Error Handling WSTG-ERRH Error messages, stack traces
Cryptography WSTG-CRYP TLS, encryption, hashing
Business Logic WSTG-BUSL Workflow bypass, file upload
Client-Side WSTG-CLNT DOM XSS, clickjacking, WebSockets
API Testing WSTG-APIT REST, GraphQL, API security

Quick Start

1. Install Dependencies

cd RAG_runner
pip install -r requirements.txt

2. Build the Database

python3 build_database.py

This will:

  • Parse all OWASP WSTG HTML files
  • Create semantic chunks for retrieval
  • Build the ChromaDB vector database

3. Start the Server

python3 -m server.http_server

Server runs on http://localhost:5004

4. Test the API

# Health check
curl http://localhost:5004/health

# Search for SQL injection testing
curl -X POST http://localhost:5004/search \
  -H "Content-Type: application/json" \
  -d '{"query": "SQL injection testing methodology"}'

# Get specific WSTG test case
curl http://localhost:5004/wstg/WSTG-INPV-05

REST API Endpoints

Endpoint Method Description
/health GET Health check
/info GET Database statistics
/list GET List all documents
/categories GET List categories and WSTG IDs
/doc/{id} GET Get document by ID
/wstg/{id} GET Get all chunks for WSTG ID
/search POST Semantic search

Search Request Body

{
  "query": "SQL injection testing",
  "n_results": 5,
  "category": "input_validation",
  "wstg_id": "WSTG-INPV-05"
}

Claude Code Integration (MCP)

Add to ~/.claude.json:

{
  "mcpServers": {
    "owasp-wstg-rag": {
      "command": "python3",
      "args": ["/path/to/OWASP_WSTG_Rag/RAG_runner/server/mcp_client.py"],
      "env": {
        "WSTG_RAG_URL": "http://localhost:5004"
      }
    }
  }
}

MCP Tools

Tool Description
search_wstg Search WSTG for testing methodologies
search_test_methodology Search for how-to testing guides
search_test_objectives Search for test objectives
get_wstg_test_case Get complete test case by WSTG ID
get_wstg_document Get document by ID
list_wstg_categories List all categories and WSTG IDs
wstg_health Health check
wstg_info Database statistics

Example Usage in Claude Code

# Search for SQL injection testing methodology
search_wstg("SQL injection testing methodology")

# Get specific test case
get_wstg_test_case("WSTG-INPV-05")

# Search within a category
search_wstg("authentication bypass", category_filter="authentication")

# Get test objectives for IDOR
search_test_objectives("IDOR insecure direct object reference")

Project Structure

OWASP_WSTG_Rag/
├── README.md
├── CLAUDE.md                    # Claude Code project guide
├── raw_data/                    # OWASP WSTG HTML source files
│   ├── 01-Information_Gathering/
│   ├── 02-Configuration_and_Deployment_Management_Testing/
│   ├── 03-Identity_Management_Testing/
│   ├── 04-Authentication_Testing/
│   ├── 05-Authorization_Testing/
│   ├── 06-Session_Management_Testing/
│   ├── 07-Input_Validation_Testing/
│   ├── 08-Testing_for_Error_Handling/
│   ├── 09-Testing_for_Weak_Cryptography/
│   ├── 10-Business_Logic_Testing/
│   ├── 11-Client-side_Testing/
│   └── 12-API_Testing/
└── RAG_runner/
    ├── build_database.py        # Main build pipeline
    ├── requirements.txt
    ├── parsers/
    │   └── wstg_parser.py       # HTML parser for WSTG
    ├── chunking/
    │   └── chunker.py           # Semantic chunking
    ├── server/
    │   ├── vector_store.py      # ChromaDB wrapper
    │   ├── http_server.py       # REST API server
    │   └── mcp_client.py        # MCP tools for Claude Code
    └── data/
        ├── processed/           # Intermediate JSON files
        └── chroma_db/           # Vector database

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    OWASP WSTG HTML Files                        │
│                      (raw_data/*.html)                          │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                     wstg_parser.py                              │
│              Parse HTML → Structured JSON                       │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                       chunker.py                                │
│              Create Semantic Chunks for RAG                     │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                   ChromaDB Vector Store                         │
│                 (data/chroma_db/)                               │
└────────────────────────────┬────────────────────────────────────┘
                             │
              ┌──────────────┴──────────────┐
              ▼                             ▼
┌──────────────────────────┐   ┌──────────────────────────┐
│    http_server.py        │   │    mcp_client.py         │
│    REST API :5004        │   │    MCP for Claude Code   │
│                          │   │                          │
│  GET  /health            │   │  search_wstg()           │
│  GET  /info              │   │  get_wstg_test_case()    │
│  GET  /wstg/{id}         │   │  search_test_methodology │
│  POST /search            │   │  list_wstg_categories()  │
└──────────────────────────┘   └──────────────────────────┘

Use Cases

AI-Assisted Penetration Testing

Integrate with Claude Code to get instant access to OWASP testing methodologies during security assessments:

User: "How do I test for SQL injection?"

Claude: [Queries WSTG RAG]
→ Returns WSTG-INPV-05 methodology with:
  - Test objectives
  - Step-by-step testing procedures
  - Example payloads
  - Tools to use

Automated Security Testing

Use the REST API to integrate WSTG methodologies into automated security pipelines:

import requests

# Get testing methodology for current test
response = requests.post('http://localhost:5004/search', json={
    'query': 'session fixation testing',
    'n_results': 3
})
methodology = response.json()['results']

Security Training

Quick reference for security testing methodologies during training or CTF challenges.

Requirements

  • Python 3.8+
  • ChromaDB
  • BeautifulSoup4
  • httpx
  • MCP SDK (for Claude Code integration)

License

This project uses content from the OWASP Web Security Testing Guide, which is licensed under Creative Commons Attribution-ShareAlike 4.0.

Related Projects

About

OWASP Web Security Testing Guide RAG system with ChromaDB, MCP for Claude Code

Resources

Readme
Activity

Stars

17 stars

Watchers

1 watching

Forks

2 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages