Hi @brynary,
I'm in the process of migrating Serenity/JS to use Qlty (see serenity-js/serenity-js#2933) and noticed an issue with coverage merging.
The new Qlty dashboard looks great, by the way, and I'd love to use it for Serenity/JS!
Context
- I use Qlty Action in the Serenity/JS TypeScript monorepo
- Automated tests are executed across multiple parallel GitHub actions
- Qlty publish partial is used to upload
lcov reports after each test job
- Qlty
complete command runs when the pipeline finishes.
- Minimal setup based on the Qlty migration docs - see config
Pipeline configuration
Test jobs are configured to publish incomplete coverage reports, for example:
test-linux-node-lts-active:
name: 'Test: Linux, LTS'
needs:
- lint
- compile
runs-on: ubuntu-24.04
permissions:
checks: write
contents: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
# ... build, run tests
- name: Qlty
uses: qltysh/qlty-action/coverage@v1.1.1
with:
oidc: true
command: publish
files: packages/*/target/coverage/lcov.info
incomplete: true
Final job configured to complete the report submission:
coverage:
name: 'Coverage'
needs:
- test-linux-node-lts-active
- test-integration
- test-integration-playwright
- test-integration-protractor
- test-integration-webdriverio
runs-on: ubuntu-24.04
permissions:
checks: write
contents: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
- name: Qlty
uses: qltysh/qlty-action/coverage@v1.1.1
with:
oidc: true
command: complete
Issue
Qlty CLI correctly detects the head SHA when running the Qlty publish action as part of the test job.
This can be seen in the Settings section of the execution log
SETTINGS
cwd: /home/runner/work/serenity-js/serenity-js
override-branch: ci/coverage-reporting
override-commit-sha: 48a1d583bae82d51fecfd4c06170ae88b133c672
incomplete: true
METADATA
CI: GitHub
Commit: 48a1d583bae82d51fecfd4c06170ae88b133c672
Pull Request: #2933
Branch: ci/coverage-reporting
Build ID: 16577701044:1
Commit Time: 2025-07-28 18:55:14 UTC
However, the Qlty complete action (see execution log):
- does not detect the Pull Request context correctly
- skips generating the override-branch and override-commit-sha config props
- errors out with a 404
Coverage report not found for commit
SETTINGS
cwd: /home/runner/work/serenity-js/serenity-js <- no branch and sha overrides
METADATA
CI: GitHub
Commit: f85f0d16761f9f022aeaedcfb0e9faf5420d0698 <- wrong commit
Pull Request: #2933
Branch: 2933/merge
Build ID: 16577701044:1
Commit Time: 2025-07-28 18:55:14 UTC
AUTHENTICATION
Auth Method: OIDC
Token: ***
COMPLETING...
ERROR
> Failed to complete coverage
>
> Caused by:
> HTTP Error 404: https://qlty.sh/api: "Coverage report not found for commit sha \"f85f0d16761f9f022aeaedcfb0e9faf5420d0698\", either because it was never created or it has expired."
Warning: Error completing coverage. Output from the Qlty CLI follows:
Warning: qlty 0.558.0 linux-x64 (b0a3def 2025-07-27)
2025-07-28T19:07:09.848702Z
Root cause
I believe the root cause of the issue is a difference in building the configuration for the Qlty CLI publish and complete commands.
For publish, this action calls buildPublishArgs, which then overrides the branch and sha for PRs:
|
let uploadArgs = await this.buildPublishArgs(); |
The complete action lacks the handling, which then leads to the error:
|
const completeArgs = ["coverage", "complete"]; |
Hi @brynary,
I'm in the process of migrating Serenity/JS to use Qlty (see serenity-js/serenity-js#2933) and noticed an issue with coverage merging.
The new Qlty dashboard looks great, by the way, and I'd love to use it for Serenity/JS!
Context
lcovreports after each test jobcompletecommand runs when the pipeline finishes.Pipeline configuration
Test jobs are configured to publish
incompletecoverage reports, for example:Final job configured to
completethe report submission:Issue
Qlty CLI correctly detects the head SHA when running the Qlty
publishaction as part of the test job.This can be seen in the
Settingssection of the execution logHowever, the Qlty
completeaction (see execution log):Coverage report not found for commitRoot cause
I believe the root cause of the issue is a difference in building the configuration for the Qlty CLI
publishandcompletecommands.For
publish, this action callsbuildPublishArgs, which then overrides the branch and sha for PRs:qlty-action/coverage/src/action.ts
Line 103 in 325e718
The
completeaction lacks the handling, which then leads to the error:qlty-action/coverage/src/action.ts
Line 171 in 325e718