Skip to content

CI: Add cirrus-ci to test linux_aarch64#22650

Merged
mattip merged 6 commits intonumpy:mainfrom
andyfaff:cirrus
Nov 28, 2022
Merged

CI: Add cirrus-ci to test linux_aarch64#22650
mattip merged 6 commits intonumpy:mainfrom
andyfaff:cirrus

Conversation

@andyfaff
Copy link
Copy Markdown
Member

Addresses a large proportion of #22640. This PR adds cibuildwheel (build+test) for linux_aarch64 on cirrus-ci.

@mattip
Copy link
Copy Markdown
Member

mattip commented Nov 23, 2022

Are the 3.9 and 3.10 builds joined into one task on purpose?

@andyfaff
Copy link
Copy Markdown
Member Author

I guess to save on firing another VM up. They can all be individual jobs if you want

@mattip
Copy link
Copy Markdown
Member

mattip commented Nov 23, 2022

Maybe a comment to say why two together and not 3 separate or 3 together (it would push the buld/test over the time limit?)

@andyfaff
Copy link
Copy Markdown
Member Author

I split into separate runs, probably easier to follow.

@mattip
Copy link
Copy Markdown
Member

mattip commented Nov 26, 2022

I went to log in to cirrus CI via my github login to create the tokens, but got this screen. As a numpy/numpy admin I am a little hesitant to let them "act on my behalf" which, as I understand it, could mean any actions I can do in the repo.
Screenshot from 2022-11-26 19-03-34

@andyfaff
Copy link
Copy Markdown
Member Author

@mattip did it give more detail under learn more? If I look at the installed app on my personal account it has the following permissions:

  • Read access to files located at .cirrus.yml
  • Read access to code, metadata, and pull requests
  • Read and write access to checks and commit statuses
  • Write access to attach content to the following external domain:
    cirrus-ci.com

These seem reasonable to me. In any case I think you need to be adding encrypted variables at https://cirrus-ci.com/github/numpy/numpy. There should be a gear/settings icon at the top RHS. You press that and you should come up with a settings page where you can enter secured variables. That's where you put the anaconda token you generated. It spits back a string that needs to be entered into the PR here

Screenshot 2022-11-27 at 11 26 57

@rgommers did this for scipy.

@mattip
Copy link
Copy Markdown
Member

mattip commented Nov 27, 2022

I added encrypted tokens from the appropriate https://anaconda.org/*/settings/access location

@mattip mattip merged commit bba7474 into numpy:main Nov 28, 2022
@mattip
Copy link
Copy Markdown
Member

mattip commented Nov 28, 2022

Thanks @andyfaff

@andyfaff
Copy link
Copy Markdown
Member Author

andyfaff commented Nov 29, 2022

@mattip

TLDR:

  • strongly advise removing the set_travis_vars function call
  • add the bash code at the end to detect whether a manual run has started and set IS_SCHEDULE_DISPATCH="true".

You asked a question:

do you know how to set the api environment variable when manually triggering a build via the + button on the gui?

But I can't seem to find it in this PR any more. This PR intentionally didn't use set_travis_vars from tools/wheels/upload_wheels.sh. There is no need to call that bash function because none of those environment variables used in that function (e.g. TRAVIS_EVENT_TYPE) will be set because it's not a TravisCI run. In fact calling set_travis_vars is probably detrimental to the upload because I the function will set IS_SCHEDULE_DISPATCH="false", IS_PUSH="false" because the TravisCI environment variables aren't present. This will undo work done here that goes to the effort of setting IS_PUSH and IS_SCHEDULE_DISPATCH in the cirrus script. I would advise removing the set_travis_vars call.

I have just been experimenting with a manual api build on my scipy fork. I created a manual build in the manner you suggested, printing out environment variables from the run. Here is the comparison from a normal push to the manual press. I have removed the environment variables that are the same between the two runs.

Normal (i.e. a PR/push/etc)

CIRRUS_CHANGE_TITLE=TST
CIRRUS_CHANGE_MESSAGE=TST
CIRRUS_COMMIT_MESSAGE=TST
CIRRUS_BUILD_SOURCE=github

Manual build (on main):

CIRRUS_CHANGE_TITLE=API build for null
CIRRUS_CHANGE_MESSAGE=API build for null
CIRRUS_COMMIT_MESSAGE=API build for null
CIRRUS_BUILD_SOURCE=api

Thus, to test for a manual build, and therefore set IS_SCHEDULE_DISPATCH="true", it may be sufficient to add the following to cirrus_general.yml (here):

if [[ $CIRRUS_BUILD_SOURCE=="api" && $CIRRUS_COMMIT_MESSAGE=="API build for null" ]]; then
    export IS_SCHEDULE_DISPATCH="true"
fi

There may be a better way of achieving this using GraphQL, but that would take a lot more experimenting.

EDIT: now I see that set_travis_vars has also been edited to take into account the CIRRUS_* env vars. However, I don't think that just checking for the presence of CIRRUS_API_CREATED is precise enough. This is because there are ways other than a manual trigger that could set that environment variable, I would advise using the bash code above as a more precise way of figuring out whether a manual run has been triggered.

charris added a commit to charris/numpy that referenced this pull request Dec 2, 2022
Backport of numpy#22650, numpy#22685, numpy#22690. The travisci builds were failing
for the Python 3.8 builds, so use cirrus instead.
@andyfaff andyfaff deleted the cirrus branch April 11, 2023 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants