Skip to content

Rust benchmarks not comparing to baseline #231

@rcrisanti

Description

@rcrisanti

It seems like for rust benchmarks the benchmark names are not being found from the previous data, even though it exists in the data.js file. You can see this even in this repo's own workflows. If you check the logs of a run of .github/workflows/ci.yml in the "Run Rust benchmark example" job at the end of the "Store benchmark result" step you will see in the generated table that the "Previous" & "Ratio" columns are empty. (For example here.)

I haven't been able to figure out why, because it seems like the data exists in the data.js file, and it is working fine for other tools. I have reproduced this in one of the repos I work on, where we have a pytest-benchmark job that works fine, but a cargo bench job that does not.

This doesn't raise an error, but it won't block a PR even if the new benchmark is over the threshold.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions