Skip to content

Reporting rendering regression on PRs #15918

@mockersf

Description

@mockersf

Good news! Since #13248 (and #15894, #15911 and #15914) CI is now checking screenshots taken during example runs on macOS and Linux for changes.

The list of examples that are executed can be found there: https://github.com/bevyengine/bevy/tree/main/.github/example-run. At the moment, only 2 examples take a screenshot: breakout and load_gltf

The macOS run is done on every push to a PR, in the merge queue, and on main. You can find an example run there: https://github.com/bevyengine/bevy/actions/runs/11339649537/job/31534799604

The Linux run is done in the merge queue and on main. You can find an example run there: https://github.com/bevyengine/bevy/actions/runs/11339392720/job/31534437159

As you may have noticed, they both fail: this is because the example load_gltf is a tiny bit time dependent for the light movement, so not fully deterministic. Failures are currently not reported or blocking for PRs.

It is possible to make examples deterministic even when relying on time by applying the patches there, or using the example-showcase tool.

You can see a list of run here: https://pixel-eagle.com/project/B04F67C0-C054-4A6F-92EC-F599FEC2FD1D

I have some improvements planned for the reporting, but now my question to you, dear contributors: which examples should be ran and checked for changes in CI?

This issue will be fixed by changing the list of screenshots compared to a fully deterministic one (either by selecting deterministic examples, or by enabling stable time with the patches), and changes being reported by a comment on PRs (not blocking merging yet)

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-Build-SystemRelated to build systems or continuous integrationA-RenderingDrawing game state to the screenS-Needs-DesignThis issue requires design work to think about how it would best be accomplished

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions