Skip to content

ci: expand unity-tests matrix to catch version-specific compile errors (2022.3, 6000.x) #1107

@dsarno

Description

@dsarno

Problem

The unity-tests.yml workflow currently runs only on Unity 2021.3.45f2. Compile errors that are version-specific (e.g. anything that diverges between 2021.3 / 2022.3 / 6000.x APIs) slip through CI and only surface when end users update the package.

Two recent incidents illustrate this:

Proposal: tiered matrix

Run a fast PR matrix and a thorough push-to-beta matrix:

Trigger Versions Wall-clock
pull_request_target (current safe-to-test flow) + in-repo PRs 2021.3.45f2 only ~3 min
push: branches: [beta] 2021.3.45f2, 2022.3.x (latest LTS), 6000.x (latest 6 LTS) ~6-8 min (parallel)
Optional nightly Widest matrix n/a

Per-PR cost stays the same as today. Per-merge-to-beta cost goes up ~3× in runner minutes, but that fires far less frequently than PRs and the existing release-gating in #1103 already blocks PyPI/version-bump on test failure — so widening the matrix at the beta gate is exactly where it has the most leverage.

Implementation sketch

In .github/workflows/unity-tests.yml, switch the matrix to use include: and condition on github.event_name/github.ref:

strategy:
  fail-fast: false
  matrix:
    include:
      - { unityVersion: "2021.3.45f2", projectPath: "TestProjects/UnityMCPTests", testMode: "editmode" }
      # Beta gate adds the wider matrix:
      - { unityVersion: "2022.3.x",    projectPath: "TestProjects/UnityMCPTests", testMode: "editmode" }
      - { unityVersion: "6000.0.x",    projectPath: "TestProjects/UnityMCPTests", testMode: "editmode" }
    exclude:
      - unityVersion: "2022.3.x"
        # only run on push to beta or workflow_call from a release pipeline
      - unityVersion: "6000.0.x"
        # ditto

(The exact gating expression depends on which form actions/runner accepts in exclude:; a conditional skip inside the steps via if: ${{ matrix.unityVersion == '2021.3.45f2' || github.ref == 'refs/heads/beta' }} is more reliable.)

License/cost notes

game-ci/unity-test-runner reuses a single UNITY_LICENSE across parallel matrix entries within one workflow run. Per-job cost is just runner minutes (~3 min × N versions, parallelized), not extra license seats.

Acceptance criteria

  • PRs touching MCPForUnity/Editor/** or TestProjects/UnityMCPTests/** still run a quick single-version test in CI.
  • Pushes to beta (and the existing workflow_call from beta-release.yml/release.yml) run the wider matrix.
  • Matrix failures on any Unity version block the existing update_unity_beta_version and publish_pypi_prerelease jobs.
  • Versions are pinned to specific patch releases for reproducibility (no 2022.3.x floating tags in CI).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions