Skip to content

Fix arguments precedence for contract tests across config, settings and CLI#2299

Merged
joelrosario merged 7 commits intomainfrom
test-opts-precedence
Feb 26, 2026
Merged

Fix arguments precedence for contract tests across config, settings and CLI#2299
joelrosario merged 7 commits intomainfrom
test-opts-precedence

Conversation

@StarKhan6368
Copy link
Copy Markdown
Contributor

@StarKhan6368 StarKhan6368 commented Feb 25, 2026

What: Fix arguments precedence for contract tests across config, settings and CLI

How:

  • Added withTestBaseURL(...) support in config implementations
  • Updated SpecmaticJUnitSupport.constructTestBaseURL() precedence logic
  • Fixed V3 config mutation paths (withTestBaseURL, test modes/filter/timeout and related config creation/update behavior)

Checklist:

  • Unit Tests
  • Build passing locally
  • Sonar Quality Gate
  • Security scans don't report any vulnerabilities
  • Documentation added/updated (share link)
  • Sample Project added/updated (share link)
  • Demo video (share link)
  • Article on Website (share link)
  • Roadmpap updated (share link)
  • Conference Talk (share link)

@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 25, 2026 •
edited
Loading

Summary

Line coverage Branch coverage

Assembly Line coverage Branch coverage
application 58.8% 52.6%
application/backwardCompatibility 77.2% 62.1%
application/mcp 84.3% 89.2%
application/validate 1.3% 0%
io/specmatic/conversions 93.1% 80.4%
io/specmatic/conversions/lenient 93.9% 79.4%
io/specmatic/core 80.1% 62.5%
io/specmatic/core/azure 0% 0%
io/specmatic/core/config 59.2% 35.8%
io/specmatic/core/config/v1 98% 25%
io/specmatic/core/config/v2 91.9% 77.5%
io/specmatic/core/config/v3 58.1% 32.2%
io/specmatic/core/config/v3/components 71.4% 40%
io/specmatic/core/config/v3/components/runOptions 73% 44.7%
io/specmatic/core/config/v3/components/services 85.5% 59.9%
io/specmatic/core/config/v3/components/settings 98.3% 63.8%
io/specmatic/core/config/v3/components/sources 82.8% 60.6%
io/specmatic/core/config/v3/specmatic 65.3% 25%
io/specmatic/core/discriminator 53.2% 43.1%
io/specmatic/core/examples/module 43.3% 24.5%
io/specmatic/core/examples/server 70.4% 35.7%
io/specmatic/core/filters 83.6% 70.1%
io/specmatic/core/fuzzy 95.8% 92.7%
io/specmatic/core/git 8.1% 12.8%
io/specmatic/core/jsonoperator 58.4% 68.6%
io/specmatic/core/jsonoperator/value 85.7% 75.9%
io/specmatic/core/lifecycle 0% ****
io/specmatic/core/log 74.5% 62%
io/specmatic/core/matchers 76.7% 62.6%
io/specmatic/core/overlay 85.7% 58.3%
io/specmatic/core/pattern 87.1% 76.3%
io/specmatic/core/pattern/config 100% ****
io/specmatic/core/pattern/regex 94% 86.5%
io/specmatic/core/report 97.7% 92.8%
io/specmatic/core/route/modules 95.8% 90%
io/specmatic/core/utilities 64.8% 56.5%
io/specmatic/core/value 81.4% 71.5%
io/specmatic/core/wsdl/parser 80.9% 41.4%
io/specmatic/core/wsdl/parser/message 88.6% 67.8%
io/specmatic/core/wsdl/parser/operation 100% 100%
io/specmatic/core/wsdl/payload 71.7% 33.3%
io/specmatic/junit5/support 0% ****
io/specmatic/loader 80.4% 50.8%
io/specmatic/mcp/constants 0% ****
io/specmatic/mcp/report 0% 0%
io/specmatic/mcp/test 0% 0%
io/specmatic/mcp/test/client 0% 0%
io/specmatic/mcp/test/client/model 0% 0%
io/specmatic/mock 61.1% 57.3%
io/specmatic/proxy 66.5% 48.6%
io/specmatic/specmatic/core 100% ****
io/specmatic/specmatic/executable 100% ****
io/specmatic/specmatic/mcp 0% ****
io/specmatic/stub 78.7% 68.9%
io/specmatic/stub/listener 96.8% 100%
io/specmatic/stub/report 98.1% 75%
io/specmatic/test 73.1% 62.8%
io/specmatic/test/asserts 92.1% 86.6%
io/specmatic/test/fixtures **** ****
io/specmatic/test/handlers 90.3% 63.5%
io/specmatic/test/listeners 86.9% 53.9%
io/specmatic/test/matchers **** ****
io/specmatic/test/reports 69% 36.4%
io/specmatic/test/reports/coverage 96.9% 53.9%
io/specmatic/test/reports/coverage/console 85.8% 81%
io/specmatic/test/reports/coverage/html 0% 0%
io/specmatic/test/reports/renderers 12.4% 14%
io/specmatic/test/utils 92.1% 82%

StarKhan6368 and others added 5 commits February 26, 2026 14:52
@StarKhan6368
Add withTestBaseUrl method to SpecmaticConfig
be40823 
- Add simple tests for the same
- Fix issues with V3 early return for modification overrides
- Add tests for V3 override methods
@StarKhan6368
Ensure precedence ordering in ContractTestSettings
2261181 
- Add simple tests for the same
@StarKhan6368
Improve precedence of testBaseUrl
2bfee73 
- Prefer config baseUrl if host/port is not explicitly specified
- Add simple tests for the same
@joelrosario @StarKhan6368
Renamed config adjuster for enabling resiliency tests
b3baf79
@StarKhan6368
Fix ContractTestSettings logic when generative is false
82beba3 
- Should update specmaticConfig with resiliency none
- Instead of resiliency positiveOnly
@joelrosario joelrosario merged commit 9f43380 into main Feb 26, 2026
3 checks passed
@joelrosario joelrosario deleted the test-opts-precedence branch February 26, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@joelrosario joelrosario joelrosario approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@StarKhan6368 @joelrosario