Skip to content

Undo doing less bench iterations under cfg(test)#6021

Merged
nical merged 1 commit intogfx-rs:trunkfrom
Wumpf:undo-limit-iterations-for-bench
Jul 24, 2024
Merged

Undo doing less bench iterations under cfg(test)#6021
nical merged 1 commit intogfx-rs:trunkfrom
Wumpf:undo-limit-iterations-for-bench

Conversation

@Wumpf
Copy link
Copy Markdown
Member

@Wumpf Wumpf commented Jul 23, 2024

Messed this up recently, was under the impression that this limits the iteration count only when running tests. Need to find another way to do that.

Still don't quite understand why that happens, I was fairly convinced that cfg(test) is not set for benchmarks in benches directory.

@Wumpf Wumpf requested a review from a team as a code owner July 23, 2024 08:00
@nical nical merged commit 9b680e6 into gfx-rs:trunk Jul 24, 2024
@teoxoy
Copy link
Copy Markdown
Member

teoxoy commented Jul 24, 2024

I think we should lower the numbers as the mac CI is now intermittently timing out.

It was close to timing out in this PR, timed out on trunk and #5714 causes it to time out 100% of the time due to the overhead of validation.

        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Computepass Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Submit Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Computepass Time)
        PASS [  78.520s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Computepass Time)
        PASS [  82.692s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Computepass Time)
        PASS [  83.268s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Submit Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Submit Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/4 computepasses x 2500 dispatches (Computepass Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/4 computepasses x 2500 dispatches (Submit Time)
        PASS [  77.072s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Submit Time)
        PASS [  75.059s] wgpu-benchmark::bench/root Computepass: Single Threaded/4 computepasses x 2500 dispatches (Computepass Time)
        PASS [  75.075s] wgpu-benchmark::bench/root Computepass: Single Threaded/4 computepasses x 2500 dispatches (Submit Time)

In this PR (https://github.com/gfx-rs/wgpu/actions/runs/10055037477/job/27791035441)

        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Computepass Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Submit Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Computepass Time)
        PASS [  70.293s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Computepass Time)
 TERMINATING [> 90.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Computepass Time)
 TERMINATING [> 90.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Submit Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Submit Time)
 TERMINATING [> 90.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Submit Time)

On trunk (https://github.com/gfx-rs/wgpu/actions/runs/10071973334/job/27843338838)

        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Computepass Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Submit Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Computepass Time)
        PASS [  77.019s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Computepass Time)
 TERMINATING [> 90.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Computepass Time)
 TERMINATING [> 90.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/1 computepasses x 10000 dispatches (Submit Time)
        SLOW [> 45.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Submit Time)
 TERMINATING [> 90.000s] wgpu-benchmark::bench/root Computepass: Single Threaded/2 computepasses x 5000 dispatches (Submit Time)

In #5714 (https://github.com/gfx-rs/wgpu/actions/runs/10071973334/job/27843338838)

@teoxoy
Copy link
Copy Markdown
Member

teoxoy commented Jul 24, 2024

I'll play around with the numbers and lower it in #5714.

@nical nical mentioned this pull request Jul 24, 2024
Merged
3 tasks
@teoxoy teoxoy mentioned this pull request Jul 24, 2024
Merged
@Wumpf Wumpf deleted the undo-limit-iterations-for-bench branch July 24, 2024 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nical nical nical approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Wumpf @teoxoy @nical