Skip to content

Handle missing-in-spec spec association for API coverage report#2354

Merged
jaydeepk merged 9 commits intomainfrom
codex/missing-in-spec-spec-association-tests
Mar 21, 2026
Merged

Handle missing-in-spec spec association for API coverage report#2354
jaydeepk merged 9 commits intomainfrom
codex/missing-in-spec-spec-association-tests

Conversation

@nashjain
Copy link
Copy Markdown
Contributor

@nashjain nashjain commented Mar 21, 2026 •
edited
Loading

What:

  • update missing-in-spec spec association to choose the closest matching spec by method and path prefix
  • add focused tests for missing-in-spec spec association in OpenApiCoverageReportInput
  • add a CTRF integration test for multi-spec execution details

Why:

  • the new spec-association heuristic needed regression coverage beyond the single /pets/search vs /owners/* case
  • CTRF HTML reads executionDetails.operations, so the multi-spec mapping needs end-to-end protection as well

How:

  • added helper-level tests covering deeper shared path prefixes, HTTP method preference, null-spec fallback, and deterministic unrelated-path fallback
  • added a multi-spec CTRF integration test that proves /pets/search is attached to the pets execution detail and not the owners execution detail
  • reused the existing OpenApiCoverageReportInput and CTRF generation path to keep the tests close to production behavior

Checklist:

  • Unit Tests
  • Build passing locally
  • Sonar Quality Gate N/A
  • Security scans don't report any vulnerabilities N/A
  • Documentation added/updated (share link) N/A
  • Sample Project added/updated (share link) N/A
  • Demo video (share link) N/A
  • Article on Website (share link) N/A
  • Roadmpap updated (share link) N/A
  • Conference Talk (share link) N/A

Issue ID:
Closes: N/A

Validation:

  • ./gradlew :junit5-support:test --tests "io.specmatic.test.reports.coverage.OpenApiCoverageReportInputTest"
  • ./gradlew :junit5-support:test --tests "io.specmatic.test.CtrfApiCoverageReportIntegrationTest"

@nashjain nashjain changed the title Expand missing-in-spec spec association coverage Handle missing-in-spec spec association for API coverage report Mar 21, 2026
@nashjain nashjain requested a review from yogeshnikam671 March 21, 2026 11:01
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 21, 2026 •
edited
Loading

Summary

Line coverage Branch coverage

Assembly Line coverage Branch coverage
application 64.9% 58.6%
application/backwardCompatibility 79.5% 65.1%
application/mcp 84.7% 76.9%
application/validate 1.2% 2.5%
io/specmatic/conversions 93.4% 78.4%
io/specmatic/conversions/lenient 97.6% 77.4%
io/specmatic/core 81.3% 62.8%
io/specmatic/core/azure 0% 0%
io/specmatic/core/config 61.6% 39.4%
io/specmatic/core/config/v1 97.9% 25%
io/specmatic/core/config/v2 91.9% 78.8%
io/specmatic/core/config/v3 56.7% 30.2%
io/specmatic/core/config/v3/components 78.9% 40%
io/specmatic/core/config/v3/components/runOptions 75.4% 47.5%
io/specmatic/core/config/v3/components/services 89.8% 59%
io/specmatic/core/config/v3/components/settings 100% 63.8%
io/specmatic/core/config/v3/components/sources 83.8% 61.5%
io/specmatic/core/config/v3/specmatic 57.6% 25%
io/specmatic/core/discriminator 52.6% 50%
io/specmatic/core/examples/module 45.7% 28.9%
io/specmatic/core/examples/server 73.8% 36.8%
io/specmatic/core/examples/source 91.6% 60%
io/specmatic/core/filters 83.9% 72.1%
io/specmatic/core/fuzzy 97.3% 93.8%
io/specmatic/core/git 9% 10.8%
io/specmatic/core/jsonoperator 68.3% 70.5%
io/specmatic/core/jsonoperator/value 86.2% 74.4%
io/specmatic/core/lifecycle 0% ****
io/specmatic/core/log 76.2% 62.3%
io/specmatic/core/matchers 86.9% 76.8%
io/specmatic/core/overlay 87.3% 58.8%
io/specmatic/core/pattern 87.5% 77.1%
io/specmatic/core/pattern/config 100% ****
io/specmatic/core/pattern/regex 90.9% 81.9%
io/specmatic/core/report 97.5% 91.6%
io/specmatic/core/route/modules 95% 87.5%
io/specmatic/core/utilities 70.3% 62.1%
io/specmatic/core/value 82.3% 69%
io/specmatic/core/wsdl/parser 81.4% 41%
io/specmatic/core/wsdl/parser/message 90% 72%
io/specmatic/core/wsdl/parser/operation 100% 100%
io/specmatic/core/wsdl/payload 81.1% 50%
io/specmatic/junit5/support 0% ****
io/specmatic/loader 84.6% 53.5%
io/specmatic/mcp/constants 0% ****
io/specmatic/mcp/report 0% 0%
io/specmatic/mcp/test 0% 0%
io/specmatic/mcp/test/client 0% 0%
io/specmatic/mcp/test/client/model 0% 0%
io/specmatic/mock 61.7% 56.5%
io/specmatic/proxy 67% 52%
io/specmatic/specmatic/core 100% ****
io/specmatic/specmatic/executable 100% ****
io/specmatic/specmatic/mcp 0% ****
io/specmatic/stub 79.3% 69.1%
io/specmatic/stub/listener 100% 100%
io/specmatic/stub/report 100% 77%
io/specmatic/test 74.2% 63.4%
io/specmatic/test/asserts 95.2% 86.6%
io/specmatic/test/handlers 89.7% 63.3%
io/specmatic/test/listeners 86.4% 53.2%
io/specmatic/test/reports 60% 33.3%
io/specmatic/test/reports/coverage 97.6% 58.3%
io/specmatic/test/reports/coverage/console 90.3% 80.9%
io/specmatic/test/reports/coverage/html 0% 0%
io/specmatic/test/reports/renderers 23.1% 25%
io/specmatic/test/utils 92.3% 80%

@nashjain nashjain requested a review from jaydeepk March 21, 2026 13:06
@jaydeepk jaydeepk merged commit 68130b7 into main Mar 21, 2026
3 checks passed
@jaydeepk jaydeepk deleted the codex/missing-in-spec-spec-association-tests branch March 21, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yogeshnikam671 yogeshnikam671 Awaiting requested review from yogeshnikam671

@jaydeepk jaydeepk Awaiting requested review from jaydeepk

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nashjain @jaydeepk