Prevent ReDoS in SSTI validation patterns#2516
Conversation
382cbf6 to
9383bd4
Compare
24d9f7c to
6eff6e7
Compare
|
@shoummu1 I've made some updates, please review / retest |
Summary of ChangesThis PR has been significantly refactored to use a manual parser approach instead of regex patterns for SSTI detection. This eliminates the ReDoS vulnerability while improving bypass resistance. Implementation ChangesReplaced regex patterns with linear-time parser:
New functions added to
New constants:
Bypass Resistance
Pre-existing LimitationThe fully-split dunder bypass ( Test Coverage
Performance
|
62ed8a9 to
040b8a9
Compare
|
@crivetimihai I've reviewed and tested the refactored implementation. Test Results
Implementation Verification✅ Parser functions present and correct:
✅ Regex patterns successfully replaced with:
✅ Bypass resistance verified:
Security Improvements Confirmed
|
3a8f8b2 to
d76fcaf
Compare
Replace regex-based SSTI detection with a linear-time manual parser
to eliminate ReDoS vulnerability while improving bypass resistance.
Changes:
- Add _iter_template_expressions() parser that correctly handles:
- Quoted strings (single and double quotes)
- Escaped characters within strings
- Nested delimiters inside quotes (e.g., "}}" in strings)
- Continues scanning after unterminated expressions (fail-closed)
- Replace _SSTI_PATTERNS regex list with:
- _SSTI_DANGEROUS_SUBSTRINGS tuple for keyword detection
- _SSTI_DANGEROUS_OPERATORS tuple for arithmetic in {{ }} and {% %}
- _SSTI_SIMPLE_TEMPLATE_PREFIXES for ${, #{, %{ expressions
- Add _has_simple_template_expression() with O(n) linear scan using rfind
- Fix type annotation for validate_parameter_length()
- Block dynamic attribute access bypasses:
- |attr filter for dynamic attribute access (with whitespace normalization)
- |selectattr, |sort, |map filters (can take attribute names)
- getattr function
- ~ operator for string concatenation (dunder name construction)
- [ bracket notation for dynamic access
- % operator for string formatting (e.g., '%c' % 95)
- attribute= parameter (blocks map/selectattr/sort attribute access)
- All escape sequences: \x, \u, \N{, \0-\7 (octal)
- Apply operator checks to both {{ }} and {% %} blocks
- Normalize whitespace around | and = before checking
Performance:
- O(n) linear scanning eliminates catastrophic backtracking
- _has_simple_template_expression uses rfind for O(n) instead of O(n²)
Security:
- Proper quote handling blocks bypasses like {{ "}}" ~ self.__class__ }}
- Escaped quote handling blocks {{ "a\"}}b" ~ self }} bypasses
- Blocks dynamic construction bypasses via string concatenation
- Blocks all escape sequence bypasses (hex, unicode, octal)
- Blocks whitespace-based bypasses around | and =
- Blocks % formatting bypasses (e.g., '%c%c' % (95,95))
- Fail-closed: continues scanning after unterminated expressions
Tests:
- Add comprehensive SSTI bypass test cases
- Add pytest.mark.timeout(30) for deterministic ReDoS detection
- Add pathological input tests for ReDoS prevention verification
Closes #2366
Co-authored-by: Shoumi <shoumimukherjee@gmail.com>
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
d76fcaf to
d69cea0
Compare
- Raise ValueError immediately on unterminated {{ or {% expressions
- Eliminates O(n²) rescan path, restoring O(n) worst-case performance
- Use consistent error message with other validation failures
- Add regression test for unterminated expression rejection
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
Changes SummaryThis PR has been rebased onto main and includes the following security hardening: ReDoS Prevention
SSTI Bypass Vectors Blocked
Performance
Fail-Closed Behavior
Tests
Other Fixes
|
Move ValueError documentation to proper Raises: section format. Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
* fix: prevent ReDoS in SSTI validation patterns
Replace regex-based SSTI detection with a linear-time manual parser
to eliminate ReDoS vulnerability while improving bypass resistance.
Changes:
- Add _iter_template_expressions() parser that correctly handles:
- Quoted strings (single and double quotes)
- Escaped characters within strings
- Nested delimiters inside quotes (e.g., "}}" in strings)
- Continues scanning after unterminated expressions (fail-closed)
- Replace _SSTI_PATTERNS regex list with:
- _SSTI_DANGEROUS_SUBSTRINGS tuple for keyword detection
- _SSTI_DANGEROUS_OPERATORS tuple for arithmetic in {{ }} and {% %}
- _SSTI_SIMPLE_TEMPLATE_PREFIXES for ${, #{, %{ expressions
- Add _has_simple_template_expression() with O(n) linear scan using rfind
- Fix type annotation for validate_parameter_length()
- Block dynamic attribute access bypasses:
- |attr filter for dynamic attribute access (with whitespace normalization)
- |selectattr, |sort, |map filters (can take attribute names)
- getattr function
- ~ operator for string concatenation (dunder name construction)
- [ bracket notation for dynamic access
- % operator for string formatting (e.g., '%c' % 95)
- attribute= parameter (blocks map/selectattr/sort attribute access)
- All escape sequences: \x, \u, \N{, \0-\7 (octal)
- Apply operator checks to both {{ }} and {% %} blocks
- Normalize whitespace around | and = before checking
Performance:
- O(n) linear scanning eliminates catastrophic backtracking
- _has_simple_template_expression uses rfind for O(n) instead of O(n²)
Security:
- Proper quote handling blocks bypasses like {{ "}}" ~ self.__class__ }}
- Escaped quote handling blocks {{ "a\"}}b" ~ self }} bypasses
- Blocks dynamic construction bypasses via string concatenation
- Blocks all escape sequence bypasses (hex, unicode, octal)
- Blocks whitespace-based bypasses around | and =
- Blocks % formatting bypasses (e.g., '%c%c' % (95,95))
- Fail-closed: continues scanning after unterminated expressions
Tests:
- Add comprehensive SSTI bypass test cases
- Add pytest.mark.timeout(30) for deterministic ReDoS detection
- Add pathological input tests for ReDoS prevention verification
Closes IBM#2366
Co-authored-by: Shoumi <shoumimukherjee@gmail.com>
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
* lint
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
* fix: enforce true fail-closed on unterminated template expressions
- Raise ValueError immediately on unterminated {{ or {% expressions
- Eliminates O(n²) rescan path, restoring O(n) worst-case performance
- Use consistent error message with other validation failures
- Add regression test for unterminated expression rejection
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
* fix: add proper Raises section to docstring for darglint
Move ValueError documentation to proper Raises: section format.
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
---------
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
Co-authored-by: Mihai Criveti <crivetimihai@gmail.com>
Signed-off-by: hughhennnelly <hughhennelly06@gmail.com>
* fix: prevent ReDoS in SSTI validation patterns
Replace regex-based SSTI detection with a linear-time manual parser
to eliminate ReDoS vulnerability while improving bypass resistance.
Changes:
- Add _iter_template_expressions() parser that correctly handles:
- Quoted strings (single and double quotes)
- Escaped characters within strings
- Nested delimiters inside quotes (e.g., "}}" in strings)
- Continues scanning after unterminated expressions (fail-closed)
- Replace _SSTI_PATTERNS regex list with:
- _SSTI_DANGEROUS_SUBSTRINGS tuple for keyword detection
- _SSTI_DANGEROUS_OPERATORS tuple for arithmetic in {{ }} and {% %}
- _SSTI_SIMPLE_TEMPLATE_PREFIXES for ${, #{, %{ expressions
- Add _has_simple_template_expression() with O(n) linear scan using rfind
- Fix type annotation for validate_parameter_length()
- Block dynamic attribute access bypasses:
- |attr filter for dynamic attribute access (with whitespace normalization)
- |selectattr, |sort, |map filters (can take attribute names)
- getattr function
- ~ operator for string concatenation (dunder name construction)
- [ bracket notation for dynamic access
- % operator for string formatting (e.g., '%c' % 95)
- attribute= parameter (blocks map/selectattr/sort attribute access)
- All escape sequences: \x, \u, \N{, \0-\7 (octal)
- Apply operator checks to both {{ }} and {% %} blocks
- Normalize whitespace around | and = before checking
Performance:
- O(n) linear scanning eliminates catastrophic backtracking
- _has_simple_template_expression uses rfind for O(n) instead of O(n²)
Security:
- Proper quote handling blocks bypasses like {{ "}}" ~ self.__class__ }}
- Escaped quote handling blocks {{ "a\"}}b" ~ self }} bypasses
- Blocks dynamic construction bypasses via string concatenation
- Blocks all escape sequence bypasses (hex, unicode, octal)
- Blocks whitespace-based bypasses around | and =
- Blocks % formatting bypasses (e.g., '%c%c' % (95,95))
- Fail-closed: continues scanning after unterminated expressions
Tests:
- Add comprehensive SSTI bypass test cases
- Add pytest.mark.timeout(30) for deterministic ReDoS detection
- Add pathological input tests for ReDoS prevention verification
Closes IBM#2366
Co-authored-by: Shoumi <shoumimukherjee@gmail.com>
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
* lint
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
* fix: enforce true fail-closed on unterminated template expressions
- Raise ValueError immediately on unterminated {{ or {% expressions
- Eliminates O(n²) rescan path, restoring O(n) worst-case performance
- Use consistent error message with other validation failures
- Add regression test for unterminated expression rejection
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
* fix: add proper Raises section to docstring for darglint
Move ValueError documentation to proper Raises: section format.
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
---------
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
Co-authored-by: Mihai Criveti <crivetimihai@gmail.com>
🐛 Bug-fix PR
📌 Summary
Fixes ReDoS (Regular Expression Denial of Service) vulnerability in Server-Side Template Injection (SSTI) validation patterns that could allow attackers to cause CPU exhaustion and service timeouts through crafted input.
🔗 Related Issue
Closes #2366
🔁 Reproduction Steps
The vulnerability could be triggered by:
{{aaaa...aaaa(unclosed brackets) to any validation endpoint🐞 Root Cause
The
_SSTI_PATTERNSlist inmcpgateway/common/validators.py(lines 95-103) used greedy quantifiers.*in regex patterns like:When given input without closing delimiters (e.g.,
{{aaaa...aaaa), the regex engine would:.*greedily to the end}}💡 Fix Description
Replaced all vulnerable
.*patterns with negated character classes that cannot backtrack:Before:
After:
Key changes:
[^}]*- matches any character except}, bounded by delimiter[^%]*- matches any character except%, bounded by delimiterThe negated character classes provide linear time complexity O(n) instead of exponential, preventing the ReDoS attack vector.
🧪 Verification
make lintmake testmake coveragepytest tests/security/test_input_validation.pyTest Results:
test_server_side_template_injection: ✅ All 8 dangerous SSTI payloads correctly rejectedtest_regex_dos_prevention: ✅ Passes with ReDoS timing checks📐 MCP Compliance (if relevant)
✅ Checklist
make black isort pre-commit)