Skip to content

BENCH scaling of a GridSearch with n_jobs#1427

Merged
ogrisel merged 6 commits intojoblib:masterfrom
tomMoral:BENCH_scaling_grid_search
Apr 19, 2023
Merged

BENCH scaling of a GridSearch with n_jobs#1427
ogrisel merged 6 commits intojoblib:masterfrom
tomMoral:BENCH_scaling_grid_search

Conversation

@tomMoral
Copy link
Copy Markdown
Contributor

Add a benchmark to compare joblib scaling for a GridSearch on synthetic data.
This benchmark allows to compare 2 versions of joblib by running it multiple times.

image

@tomMoral tomMoral marked this pull request as ready for review April 18, 2023 17:34
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 18, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.06 ⚠️

Comparison is base (fc6c4ce) 94.66% compared to head (cba3957) 94.61%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1427      +/-   ##
==========================================
- Coverage   94.66%   94.61%   -0.06%     
==========================================
  Files          45       44       -1     
  Lines        6918     7205     +287     
==========================================
+ Hits         6549     6817     +268     
- Misses        369      388      +19     

see 6 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Copy Markdown
Member

@GaelVaroquaux GaelVaroquaux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

The bump between 6 and 8 jobs is quite curious.

@ogrisel
Copy link
Copy Markdown
Contributor

ogrisel commented Apr 19, 2023

Thanks, here is what I get on my laptop with 4 performance cores and 4 efficiency cores.

image

I cannot reproduce the plateauing effect in thomas' plot.

@ogrisel
Copy link
Copy Markdown
Contributor

ogrisel commented Apr 19, 2023

Here is the same result with a ylim to start at 0.

image

I find it less misleading.

@ogrisel ogrisel force-pushed the BENCH_scaling_grid_search branch from 7c2ff63 to 85b1d26 Compare April 19, 2023 07:55
@ogrisel
Copy link
Copy Markdown
Contributor

ogrisel commented Apr 19, 2023

I messed up with git, but I finally pushed a simpler ylim.

@ogrisel ogrisel merged commit a4e1d71 into joblib:master Apr 19, 2023
@ogrisel
Copy link
Copy Markdown
Contributor

ogrisel commented Apr 19, 2023

I think this kind of plots would be more informative as speed-ups factor vs a chosen baseline (e.g. the slowest or the single thread result from the current branch) but this is already useful as it is so I merged.

@tomMoral tomMoral deleted the BENCH_scaling_grid_search branch April 19, 2023 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants