Skip to content

Accumulation of blocked GitHub CI jobs #957

@emanuel-schmid

Description

@emanuel-schmid

Starting on September 26 there was a nearly contiguous series of failing CI (Unit testing) actions.
The jobs usually failed after 6 hours stalling at

2024-09-26T08:46:55.4880563Z ##[group]micromamba list
2024-09-26T08:46:55.4882034Z [command]/home/runner/micromamba-bin/micromamba list -r /home/runner/micromamba -n climada_env_3.10
2024-09-26T14:29:15.9153370Z ##[error]The operation was canceled.

without further information.

According to actions/runner#1326 (comment) this may be related to memory limitations.
As we all know the environment creation requires a lot of memory ...

It is very unlikely that an environment change is the root cause for this burst of stallings, as there have been no changes in the environment around that time, neither in the requirement file nor in the environment itself.

For a proper investigation we would have to set a self-hosted runner in place with the very likely outcome that it won't reproduce.
As an immediate step I'll set timeouts for the whole job to 20 minutes as all succeeding jobs are below that.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions