Skip to content

v1.1.1 - Command: complete broken for Pull Requests #119

Description

@jan-molak

Hi @brynary,

I'm in the process of migrating Serenity/JS to use Qlty (see serenity-js/serenity-js#2933) and noticed an issue with coverage merging.

The new Qlty dashboard looks great, by the way, and I'd love to use it for Serenity/JS!

Context

  • I use Qlty Action in the Serenity/JS TypeScript monorepo
  • Automated tests are executed across multiple parallel GitHub actions
  • Qlty publish partial is used to upload lcov reports after each test job
  • Qlty complete command runs when the pipeline finishes.
  • Minimal setup based on the Qlty migration docs - see config

Pipeline configuration

Test jobs are configured to publish incomplete coverage reports, for example:

  test-linux-node-lts-active:
    name: 'Test: Linux, LTS'
    needs:
      - lint
      - compile
    runs-on: ubuntu-24.04
    permissions:
      checks: write
      contents: read
      id-token: write
    steps:
      - name: Checkout repository
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
      # ... build, run tests
      - name: Qlty
        uses: qltysh/qlty-action/coverage@v1.1.1
        with:
          oidc: true
          command: publish
          files: packages/*/target/coverage/lcov.info
          incomplete: true

Final job configured to complete the report submission:

  coverage:
    name: 'Coverage'
    needs:
      - test-linux-node-lts-active
      - test-integration
      - test-integration-playwright
      - test-integration-protractor
      - test-integration-webdriverio
    runs-on: ubuntu-24.04
    permissions:
      checks: write
      contents: read
      id-token: write
    steps:
      - name: Checkout repository
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
      - name: Qlty
        uses: qltysh/qlty-action/coverage@v1.1.1
        with:
          oidc: true
          command: complete

Issue

Qlty CLI correctly detects the head SHA when running the Qlty publish action as part of the test job.
This can be seen in the Settings section of the execution log

 SETTINGS 

    cwd: /home/runner/work/serenity-js/serenity-js
    override-branch: ci/coverage-reporting
    override-commit-sha: 48a1d583bae82d51fecfd4c06170ae88b133c672
    incomplete: true

 METADATA 

    CI: GitHub
    Commit: 48a1d583bae82d51fecfd4c06170ae88b133c672
    Pull Request: #2933
    Branch: ci/coverage-reporting
    Build ID: 16577701044:1
    Commit Time: 2025-07-28 18:55:14 UTC

However, the Qlty complete action (see execution log):

  • does not detect the Pull Request context correctly
  • skips generating the override-branch and override-commit-sha config props
  • errors out with a 404 Coverage report not found for commit
SETTINGS 

    cwd: /home/runner/work/serenity-js/serenity-js          <- no branch and sha overrides

 METADATA 

    CI: GitHub
    Commit: f85f0d16761f9f022aeaedcfb0e9faf5420d0698        <- wrong commit
    Pull Request: #2933
    Branch: 2933/merge
    Build ID: 16577701044:1
    Commit Time: 2025-07-28 18:55:14 UTC

 AUTHENTICATION 

    Auth Method: OIDC
    Token: ***

 COMPLETING... 


   ERROR   

 > Failed to complete coverage
 > 
 > Caused by:
 >     HTTP Error 404: https://qlty.sh/api: "Coverage report not found for commit sha \"f85f0d16761f9f022aeaedcfb0e9faf5420d0698\", either because it was never created or it has expired."
Warning: Error completing coverage. Output from the Qlty CLI follows:
Warning: qlty 0.558.0 linux-x64 (b0a3def 2025-07-27)
2025-07-28T19:07:09.848702Z

Root cause

I believe the root cause of the issue is a difference in building the configuration for the Qlty CLI publish and complete commands.

For publish, this action calls buildPublishArgs, which then overrides the branch and sha for PRs:

let uploadArgs = await this.buildPublishArgs();

The complete action lacks the handling, which then leads to the error:

const completeArgs = ["coverage", "complete"];

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions