Skip to content

chore(ci): use macOS to test ML dependencies#3774

Merged
ariostas merged 3 commits intoscikit-hep:mainfrom
ariostas:fix_ml_dependencies
Dec 19, 2025
Merged

chore(ci): use macOS to test ML dependencies#3774
ariostas merged 3 commits intoscikit-hep:mainfrom
ariostas:fix_ml_dependencies

Conversation

@ariostas
Copy link
Copy Markdown
Member

We changed the ML requirements to use tensorflow-cpu instead of tensorflow since it was downloading a bunch of extra stuff and running out of storage. However, tensorflow-cpu doesn't build wheels for macOS, so the integration tests are failing. Since this requirement file could be used for people as a guideline, it's good to fix it here. Thanks to @ikrommyd for pointing this out.

@ariostas ariostas requested review from ianna and ikrommyd December 17, 2025 15:16
Copy link
Copy Markdown
Member

@ianna ianna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ariostas - this is great! Thank you. Please merge it. Thanks!

@codecov
Copy link
Copy Markdown

codecov bot commented Dec 17, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.73%. Comparing base (d980f98) to head (8feea02).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Copy Markdown

The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR3774

@ariostas ariostas marked this pull request as draft December 17, 2025 15:39
@ariostas ariostas force-pushed the fix_ml_dependencies branch from 06fcc3f to 0851fd4 Compare December 17, 2025 15:47
@ariostas ariostas requested a review from ikrommyd December 17, 2025 15:49
@ariostas ariostas marked this pull request as ready for review December 17, 2025 15:53
@ariostas
Copy link
Copy Markdown
Member Author

Looking a bit more into it, it seems like torch comes with even more bloated dependencies. It end's up installing a bunch of cuda stuff, and I don't fully understand why.

@ikrommyd
Copy link
Copy Markdown
Collaborator

ikrommyd commented Dec 17, 2025

Looking a bit more into it, it seems like torch comes with even more bloated dependencies. It end's up installing a bunch of cuda stuff, and I don't fully understand why.

oh yeah, on linux pip install torch gets you full cuda support, you need to specify pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cpu to get the cpu-only build. Check out the installation matrix here: https://pytorch.org/get-started/locally/ . I don't know whether you can specify index url in requirements txts though.

@ariostas
Copy link
Copy Markdown
Member Author

Seems like we can specify it in the requirements.txt, but you can't specify it for a single package: pypa/pip#12233

@ikrommyd
Copy link
Copy Markdown
Collaborator

Will this cause problems with other packages? I'd hope that the url is "pure".

@ariostas ariostas marked this pull request as draft December 18, 2025 19:11
@ariostas
Copy link
Copy Markdown
Member Author

I'll mark this as a draft while we decide what to do. In the meeting we discussed potentially switching to conda environments or to pixi.

@ikrommyd
Copy link
Copy Markdown
Collaborator

ikrommyd commented Dec 18, 2025

I'll mark this as a draft while we decide what to do. In the meeting we discussed potentially switching to conda environments or to pixi.

it may not be bad to have this or a patch at the integration tests repo (whatever y'all prefer) in the meantime just for the sake of having the integration tests ci pass.

@ariostas ariostas marked this pull request as ready for review December 19, 2025 15:16
@ariostas
Copy link
Copy Markdown
Member Author

Yeah, that's a good point. How for now we just switch to testing ML on macOS (which doesn't have the extra nvidia bloat) and revert to the original requirements?

@ikrommyd
Copy link
Copy Markdown
Collaborator

ikrommyd commented Dec 19, 2025

Yeah, that's a good point. How for now we just switch to testing ML on macOS (which doesn't have the extra nvidia bloat) and revert to the original requirements?

That does sound good to me. Actually, even jax may bring in bloat on linux but I don't remember off the top of my head.

@ikrommyd
Copy link
Copy Markdown
Collaborator

ikrommyd commented Dec 19, 2025

I think this needs a title change now to reflect what it actually does but should be good otherwise

@ariostas ariostas changed the title chore(ci): use tensorflow-cpu only for Linux chore(ci): use macOS to test ML dependencies Dec 19, 2025
@ariostas ariostas merged commit 557f044 into scikit-hep:main Dec 19, 2025
38 of 40 checks passed
@ikrommyd
Copy link
Copy Markdown
Collaborator

ikrommyd commented Dec 20, 2025

We're fully green over at integration tests again :)
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants