aarch + ppc64le again for 0.22 by hmaarrfk · Pull Request #113 · conda-forge/scikit-learn-feedstock

hmaarrfk · 2019-12-07T14:36:55Z

Checklist

Used a fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

conda-forge-linter · 2019-12-07T14:37:02Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

hmaarrfk · 2019-12-07T20:33:23Z

This is very likely ready to go. Unfortunately, I have to recommend skipping the tests for PPC64LE and AARCH64. The build time is quite variable. Sometimes, the CIs can run through the whole build + test procedure in 23 minutes, other times, it timesout after only 10% of the tests. This doesn't seem related to the test suite, since it won't even finish the build phase until 28 mins...

hmaarrfk · 2019-12-07T21:09:10Z

See odd timing failure here https://cloud.drone.io/conda-forge/scikit-learn-feedstock/36/2/2

…ertion.patch

conda-forge-linter · 2019-12-08T00:28:30Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.

Here's what I've got...

For recipe:

Failed to even lint the recipe, probably because of a conda-smithy bug 😢. This likely indicates a problem in your meta.yaml, though. To get a traceback to help figure out what's going on, install conda-smithy and run conda smithy recipe-lint . from the recipe directory.

conda-forge-linter · 2019-12-08T00:30:04Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

hmaarrfk · 2019-12-08T01:06:47Z

finally:

the variance on the compilation + test times is insane.

ogrisel · 2019-12-09T15:12:07Z

the variance on the compilation + test times is insane.

Do you have any idea why? Noisy neighbors using the same hardware in concurrently running containers without proper cgroup quotas for CPU / disk / RAM resources maybe?

hmaarrfk · 2019-12-09T17:52:04Z

Not too sure. We can probably ping drone.

But for now. It has been passing consistently. Would you like to merge?

I the remaining failures are all legitimately issues that I reported.

rth

Minor comment otherwise LGTM. Thanks @hmaarrfk !

rth · 2019-12-09T20:32:44Z

recipe/meta.yaml

+    # build the C files in parallel. This helps beat the
+    # time limit consistently on drone.io
+    - {{ PYTHON }} setup.py build_ext -j{{ CPU_COUNT }}
+    - {{ PYTHON }} -m pip install . -vv


Do we want to keep --no-deps?

I'm 90 percent sure conda forge does this in their pip config

A while back, I went on this hunt, and found that they isolate the environment:
scikit-image/scikit-image#4261 (comment)

Actually it's done in conda-build, but yeah we don't need it here.

https://github.com/conda/conda-build/pull/3271/files

https://github.com/conda/conda-build/blob/master/CHANGELOG.txt#L577

Thanks for the confirmation!

rth

Thank you @hmaarrfk, this is great!

Attempt to fix aarch again

1ea8f8b

hmaarrfk requested review from adrinjalali, amueller, astaric, jakirkham, jnothman, lesteve, ocefpaf, ogrisel and rth as code owners December 7, 2019 14:36

hmaarrfk added 3 commits December 7, 2019 09:37

Add the patch

82098ef

Run tests in parallel

2d5cb27

Depend on pytest-xdist for tests

23999cf

hmaarrfk mentioned this pull request Dec 7, 2019

test_sag_regressor_computed_correctly failure on Python 3.7 Linux Azure scikit-learn/scikit-learn#15818

Closed

hmaarrfk added 9 commits December 7, 2019 10:54

Skip test_sag_regressor_computed_correctly

3409422

Run with verbose again

1dd071d

Try testing with verbose and multiple CPUs again

0b6d2b1

Use pytest-timeout as well

8bc3fd5

Try serial builds with timeout

5bf1bfe

Avoid creating too many test workers

3b1c0d4

remove pytest sugar

9fc0835

Remove square bracket selector

c3c7ddc

Skip an other test on aarch64

a63207d

hmaarrfk added 4 commits December 7, 2019 16:11

Attempt to build extentions with parallel CPUs for speedup

99caebe

Skip an other test, explain why

1026d2e

Unify test command between all architectures

f0cce06

Try to change the quotes for windows

71d4a29

hmaarrfk mentioned this pull request Dec 7, 2019

test_warm_start_yields_identical_results can timeout on aarch64 scikit-learn/scikit-learn#15823

Closed

hmaarrfk added 5 commits December 7, 2019 17:52

Skip an other test on aarch64

e82683d

Delete 15775-Increases-tol-for-check_pca_float_dtype_preservation-ass…

3632acd

…ertion.patch

Update meta.yaml

4c4a890

Lint

7a28b1f

skip even more tess

0b184e8

Fixup recipe

1d35eba

hmaarrfk changed the title ~~Attempt to fix aarch again for 0.22~~ aarch + ppc64le again for 0.22 Dec 8, 2019

hmaarrfk added 6 commits December 7, 2019 20:08

Bump build number

3b1133a

Try to include test_early_stopping_regression

5fac7e4

Update meta.yaml

912039e

set OMP NUM THREADS

b7b23c6

try to run the tests that were really slow before OMP setting

59063cb

cleanup the recipe

7514cf5

rth approved these changes Dec 9, 2019

View reviewed changes

rth approved these changes Dec 10, 2019

View reviewed changes

rth merged commit 393ae35 into conda-forge:master Dec 10, 2019

This was referenced Dec 10, 2019

Perhaps, test test_estimate_bandwidth_1sample can be already unmuted for aarch64 and ppc64le #106

Closed

Unignore test test_estimate_bandwidth_1sample for aarch64 and ppc64le #107

Closed

Uh oh!

Conversation

hmaarrfk commented Dec 7, 2019

Uh oh!

conda-forge-linter commented Dec 7, 2019

Uh oh!

hmaarrfk commented Dec 7, 2019

Uh oh!

hmaarrfk commented Dec 7, 2019

Uh oh!

conda-forge-linter commented Dec 8, 2019

Uh oh!

conda-forge-linter commented Dec 8, 2019

Uh oh!

hmaarrfk commented Dec 8, 2019

Uh oh!

ogrisel commented Dec 9, 2019

Uh oh!

hmaarrfk commented Dec 9, 2019

Uh oh!

rth left a comment

Choose a reason for hiding this comment

Uh oh!

rth Dec 9, 2019

Choose a reason for hiding this comment

Uh oh!

hmaarrfk Dec 10, 2019

Choose a reason for hiding this comment

Uh oh!

hmaarrfk Dec 10, 2019

Choose a reason for hiding this comment

Uh oh!

jakirkham Dec 10, 2019

Choose a reason for hiding this comment

Uh oh!

hmaarrfk Dec 10, 2019

Choose a reason for hiding this comment

Uh oh!

hmaarrfk Dec 10, 2019

Choose a reason for hiding this comment

Uh oh!

rth Dec 10, 2019

Choose a reason for hiding this comment

Uh oh!

rth left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants