Skip to content

Use coverage status to determine missed and not-implemented operation counts#2356

Merged
nashjain merged 2 commits intomainfrom
exclude_wip_ops_from_missing_ops_counts
Mar 22, 2026
Merged

Use coverage status to determine missed and not-implemented operation counts#2356
nashjain merged 2 commits intomainfrom
exclude_wip_ops_from_missing_ops_counts

Conversation

@jaydeepk
Copy link
Copy Markdown
Contributor

@jaydeepk jaydeepk commented Mar 22, 2026

What:

Use coverage status, rather than raw test results, when computing missed and not-implemented operation counts in the OpenAPI coverage report.

Why:

WIP operations were being rendered as WIP in console and CTRF reports, but still counted as missed or not implemented because the aggregate counters were based on raw MissingInSpec / NotImplemented results. This made governance counts inconsistent with the final reported coverage status.

How:

Update coverage aggregation to derive both missedOperations and notImplementedOperations from getCoverageStatus(). Add regression tests covering WIP missing-in-spec and WIP not-implemented scenarios, including cases with discovered application APIs and documented spec endpoints.

Checklist:

  • Unit Tests
  • Build passing locally
  • Sonar Quality Gate N/A
  • Security scans don't report any vulnerabilities N/A
  • Documentation added/updated (share link) N/A
  • Sample Project added/updated (share link) N/A
  • Demo video (share link) N/A
  • Article on Website (share link) N/A
  • Roadmpap updated (share link) N/A
  • Conference Talk (share link) N/A

Issue ID:
Closes: N/A

@jaydeepk
Use coverage status to determine missed and not-implemented operation…
f14817f 
… counts

This ensures that WIP ops are not counted as MissingInSpec.
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 22, 2026 •
edited
Loading

Summary

Line coverage Branch coverage

Assembly Line coverage Branch coverage
application 64.9% 58.6%
application/backwardCompatibility 79.5% 65.1%
application/mcp 84.7% 76.9%
application/validate 1.2% 2.5%
io/specmatic/conversions 93.4% 78.4%
io/specmatic/conversions/lenient 97.6% 77.4%
io/specmatic/core 81.3% 62.8%
io/specmatic/core/azure 0% 0%
io/specmatic/core/config 61.6% 39.4%
io/specmatic/core/config/v1 97.9% 25%
io/specmatic/core/config/v2 91.9% 78.8%
io/specmatic/core/config/v3 56.7% 30.2%
io/specmatic/core/config/v3/components 78.9% 40%
io/specmatic/core/config/v3/components/runOptions 75.4% 47.5%
io/specmatic/core/config/v3/components/services 89.8% 59%
io/specmatic/core/config/v3/components/settings 100% 63.8%
io/specmatic/core/config/v3/components/sources 83.8% 61.5%
io/specmatic/core/config/v3/specmatic 57.6% 25%
io/specmatic/core/discriminator 52.6% 50%
io/specmatic/core/examples/module 45.7% 28.9%
io/specmatic/core/examples/server 73.8% 36.8%
io/specmatic/core/examples/source 91.6% 60%
io/specmatic/core/filters 83.9% 72.1%
io/specmatic/core/fuzzy 97.3% 93.8%
io/specmatic/core/git 9% 10.8%
io/specmatic/core/jsonoperator 68.3% 70.5%
io/specmatic/core/jsonoperator/value 86.2% 74.4%
io/specmatic/core/lifecycle 0% ****
io/specmatic/core/log 76.2% 62.3%
io/specmatic/core/matchers 86.9% 76.8%
io/specmatic/core/overlay 87.3% 58.8%
io/specmatic/core/pattern 87.5% 77.1%
io/specmatic/core/pattern/config 100% ****
io/specmatic/core/pattern/regex 94.5% 87.5%
io/specmatic/core/report 97.5% 91.6%
io/specmatic/core/route/modules 95% 87.5%
io/specmatic/core/utilities 70.3% 62.1%
io/specmatic/core/value 82.3% 69%
io/specmatic/core/wsdl/parser 81.4% 41%
io/specmatic/core/wsdl/parser/message 90% 72%
io/specmatic/core/wsdl/parser/operation 100% 100%
io/specmatic/core/wsdl/payload 81.1% 50%
io/specmatic/junit5/support 0% ****
io/specmatic/loader 84.6% 53.5%
io/specmatic/mcp/constants 0% ****
io/specmatic/mcp/report 0% 0%
io/specmatic/mcp/test 0% 0%
io/specmatic/mcp/test/client 0% 0%
io/specmatic/mcp/test/client/model 0% 0%
io/specmatic/mock 61.7% 56.5%
io/specmatic/proxy 67% 52%
io/specmatic/specmatic/core 100% ****
io/specmatic/specmatic/executable 100% ****
io/specmatic/specmatic/mcp 0% ****
io/specmatic/stub 79.3% 69.1%
io/specmatic/stub/listener 100% 100%
io/specmatic/stub/report 100% 77%
io/specmatic/test 74.2% 63.6%
io/specmatic/test/asserts 95.2% 86.6%
io/specmatic/test/handlers 89.7% 63.3%
io/specmatic/test/listeners 86.4% 53.2%
io/specmatic/test/reports 60% 33.3%
io/specmatic/test/reports/coverage 97.6% 55.1%
io/specmatic/test/reports/coverage/console 90.9% 82.7%
io/specmatic/test/reports/coverage/html 0% 0%
io/specmatic/test/reports/renderers 20% 24.1%
io/specmatic/test/utils 92.3% 80%

@nashjain nashjain merged commit f26ee72 into main Mar 22, 2026
3 checks passed
@nashjain nashjain deleted the exclude_wip_ops_from_missing_ops_counts branch March 22, 2026 07:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jaydeepk @nashjain