Skip to content

Implement lenient tests in SpecmaticJUnitSupport#2306

Merged
joelrosario merged 2 commits intomainfrom
lenient-test
Feb 25, 2026
Merged

Implement lenient tests in SpecmaticJUnitSupport#2306
joelrosario merged 2 commits intomainfrom
lenient-test

Conversation

@StarKhan6368
Copy link
Copy Markdown
Contributor

@StarKhan6368 StarKhan6368 commented Feb 25, 2026

What: Implement lenient tests in SpecmaticJUnitSupport

How:

  • When lenient mode is activated, invalid examples are filtered out from scenarios
  • Implement a method in TestHooks to relay example errors

Checklist:

  • Unit Tests
  • Build passing locally
  • Sonar Quality Gate
  • Security scans don't report any vulnerabilities
  • Documentation added/updated (share link)
  • Sample Project added/updated (share link)
  • Demo video (share link)
  • Article on Website (share link)
  • Roadmpap updated (share link)
  • Conference Talk (share link)

@StarKhan6368
Implement lenient tests in SpecmaticJUnitSupport
df8c58e 
- When lenient mode is enabled invalid examples are filtered out
- Add method to TestHooks to relay example errors
- Add simple tests for the same
@StarKhan6368 StarKhan6368 marked this pull request as ready for review February 25, 2026 14:26
@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 25, 2026

Summary

Line coverage Branch coverage

Assembly Line coverage Branch coverage
application 58.7% 51.8%
application/backwardCompatibility 77.2% 62.1%
application/mcp 84.3% 89.2%
application/validate 1.3% 0%
io/specmatic/conversions 93.1% 80.4%
io/specmatic/conversions/lenient 93.9% 79.4%
io/specmatic/core 80.2% 62.7%
io/specmatic/core/azure 0% 0%
io/specmatic/core/config 59.2% 35.2%
io/specmatic/core/config/v1 98% 25%
io/specmatic/core/config/v2 91.9% 77.5%
io/specmatic/core/config/v3 51.6% 26.7%
io/specmatic/core/config/v3/components 71.4% 40%
io/specmatic/core/config/v3/components/runOptions 73% 44.7%
io/specmatic/core/config/v3/components/services 58.6% 50%
io/specmatic/core/config/v3/components/settings 98.3% 50%
io/specmatic/core/config/v3/components/sources 80% 59%
io/specmatic/core/config/v3/specmatic 65.3% 25%
io/specmatic/core/discriminator 53.2% 43.1%
io/specmatic/core/examples/module 43.3% 24.5%
io/specmatic/core/examples/server 70.4% 35.7%
io/specmatic/core/filters 83.6% 70.1%
io/specmatic/core/fuzzy 95.8% 92.7%
io/specmatic/core/git 8.1% 12.8%
io/specmatic/core/jsonoperator 58.4% 68.6%
io/specmatic/core/jsonoperator/value 85.7% 75.9%
io/specmatic/core/lifecycle 0% ****
io/specmatic/core/log 74.3% 61.1%
io/specmatic/core/matchers 76.7% 62.6%
io/specmatic/core/overlay 85.7% 58.3%
io/specmatic/core/pattern 87.1% 76.3%
io/specmatic/core/pattern/config 100% ****
io/specmatic/core/pattern/regex 94% 86.5%
io/specmatic/core/report 97.7% 92.8%
io/specmatic/core/route/modules 95.8% 90%
io/specmatic/core/utilities 64.8% 56.5%
io/specmatic/core/value 81.4% 71.5%
io/specmatic/core/wsdl/parser 80.9% 41.4%
io/specmatic/core/wsdl/parser/message 88.6% 67.8%
io/specmatic/core/wsdl/parser/operation 100% 100%
io/specmatic/core/wsdl/payload 71.7% 33.3%
io/specmatic/junit5/support 0% ****
io/specmatic/loader 80.4% 50.8%
io/specmatic/mcp/constants 0% ****
io/specmatic/mcp/report 0% 0%
io/specmatic/mcp/test 0% 0%
io/specmatic/mcp/test/client 0% 0%
io/specmatic/mcp/test/client/model 0% 0%
io/specmatic/mock 61.1% 57.3%
io/specmatic/proxy 66.5% 48.6%
io/specmatic/specmatic/core 100% ****
io/specmatic/specmatic/executable 100% ****
io/specmatic/specmatic/mcp 0% ****
io/specmatic/stub 78.4% 69.4%
io/specmatic/stub/listener 96.8% 90%
io/specmatic/stub/report 98.1% 75%
io/specmatic/test 73% 61.7%
io/specmatic/test/asserts 92.1% 86.6%
io/specmatic/test/fixtures **** ****
io/specmatic/test/handlers 90.3% 63.5%
io/specmatic/test/listeners 84.2% 54%
io/specmatic/test/matchers **** ****
io/specmatic/test/reports 69% 36.4%
io/specmatic/test/reports/coverage 96.9% 53.9%
io/specmatic/test/reports/coverage/console 85.8% 81%
io/specmatic/test/reports/coverage/html 0% 0%
io/specmatic/test/reports/renderers 12.4% 14%
io/specmatic/test/utils 92.1% 82%

@joelrosario joelrosario self-requested a review February 25, 2026 15:25
@joelrosario joelrosario merged commit 6e0f7ea into main Feb 25, 2026
1 of 3 checks passed
@joelrosario joelrosario deleted the lenient-test branch February 25, 2026 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@joelrosario joelrosario joelrosario approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@StarKhan6368 @joelrosario