Skip to content

Build once per platform & use ccache#7756

Draft
stefanv wants to merge 33 commits intoscikit-image:mainfrom
stefanv:cache-meson-build-dir
Draft

Build once per platform & use ccache#7756
stefanv wants to merge 33 commits intoscikit-image:mainfrom
stefanv:cache-meson-build-dir

Conversation

@stefanv
Copy link
Copy Markdown
Member

@stefanv stefanv commented Mar 15, 2025

See if we can speed up builds by caching meson build dir

@stefanv stefanv added the 🤖 type: Infrastructure CI, packaging, tools and automation label Mar 15, 2025
@stefanv stefanv force-pushed the cache-meson-build-dir branch from 43c6aa3 to f6c16d0 Compare March 15, 2025 01:00
@mdhaber
Copy link
Copy Markdown
Contributor

mdhaber commented Mar 15, 2025

It looks like it's successful at restoring the cache, but it's not faster to build. It still counts from 1 to 165. Is it using the cache?

In retrospect, sorry for suggesting caching the build folder! I was just interested in the idea of caching the build, but knew I didn't know how to do it.

@stefanv stefanv marked this pull request as draft March 15, 2025 03:26
@stefanv
Copy link
Copy Markdown
Member Author

stefanv commented Mar 15, 2025

Timestamps may be a problem. Not sure meson cache is meant to be restored like this.

@lagru
Copy link
Copy Markdown
Member

lagru commented Mar 16, 2025

Could ccache be another option (in case you haven't thought of it already)?

@stefanv
Copy link
Copy Markdown
Member Author

stefanv commented Mar 16, 2025

Yes, that's what I realized we'd have to do! You can't cache meson this way because of timestamps.

@stefanv stefanv changed the title Cache meson build dir Cache builds using ccache Mar 17, 2025
@stefanv stefanv force-pushed the cache-meson-build-dir branch from 1192698 to 4886285 Compare March 17, 2025 05:14
@stefanv
Copy link
Copy Markdown
Member Author

stefanv commented Mar 17, 2025

We should not be building N times for a given Python version on a given platform. Build once, test several configurations?

@mdhaber
Copy link
Copy Markdown
Contributor

mdhaber commented Mar 17, 2025

What configurations do you have other than different platforms/Python versions? Different versions of other dependencies?

(That said, this will already be a huge improvement, so besides getting the cache working, optimizations could be a follow-up.)

@stefanv
Copy link
Copy Markdown
Member Author

stefanv commented Mar 17, 2025

For any given platform, we build with a known set of dependencies. But, you're right, that can be a secondary optimization.

@stefanv
Copy link
Copy Markdown
Member Author

stefanv commented Mar 18, 2025

It doesn't look like ccache helped here 🤔

@stefanv stefanv force-pushed the cache-meson-build-dir branch from 0f5a1f0 to 2665540 Compare March 19, 2025 08:01
@stefanv
Copy link
Copy Markdown
Member Author

stefanv commented Mar 19, 2025

OK, the caching is roughly working. Takes a build down from ~4.5mins to ~1min. Will need to adjust the CI a bit, because looks like the caching is too granular. E.g., the pre-build still rebuilds from source, despite having access to a cache (hits: <2%).

@stefanv stefanv force-pushed the cache-meson-build-dir branch from 5e6d141 to 5617def Compare March 19, 2025 18:42
@stefanv stefanv force-pushed the cache-meson-build-dir branch from 6cbd4d8 to 93a9725 Compare March 19, 2025 19:07
@lagru lagru self-requested a review March 20, 2025 18:24
@stefanv stefanv changed the title Cache builds using ccache Build once per platform & use ccache Apr 23, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 7, 2026

Hello scikit-image core devs! There hasn't been any activity on this PR for more than 180 days. I have marked it as "dormant" to make it easy to find.

To our contributors, thank you for your contribution and apologies if this contribution fell through the cracks! Hopefully this ping will help bring some fresh attention to the review. If you need help, you can always reach out on our forum

If you think that this PR is no longer relevant, you may close it, or we may do it at some point (either way, it will be done manually). If you think the PR is valuable but you no longer have the bandwidth to update it, please let us know, so that someone can take it over. 🙏

@github-actions github-actions bot added the 😴 Dormant No recent activity label Mar 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🤖 type: Infrastructure CI, packaging, tools and automation 😴 Dormant No recent activity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants