Skip to content

Improve benchmarks #4146

@wellwelwel

Description

@wellwelwel

Currently the benchmarks report results that can vary constantly with GitHub Actions availability, suffer from environment variations (or any external factor related to the environment in which they are being executed).

Personally, I like to work with ratio-based benchmarks, where there is a baseline and a comparison for each benchmark run, and the comparison is made based on the ratios, not in the execution itself.

In MySQL2 this is a bit more complicated due to the MySQL Server (more external factors), but I believe we can explore something viable and even compare/document the results between other drivers.

Also, workflows that require secrets don't run properly on PRs from forks (even if it's secrets.GITHUB_TOKEN).

Furthermore, the benchmark report only negative cases. It's also interesting to have positive feedback.

Related

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions