Skip to content

Conversation

@delan
Copy link
Member

@delan delan commented Nov 25, 2025

the runner timeout jobs are a helpful safeguard against bugs (or github infra failures) that cause jobs to not get picked up by self-hosted runners, but all they do is wait two minutes and cancel the run, clogging up our github-hosted runner capacity (limited to 60).

this patch removes the timeout jobs, based on the expectation that our CI system is reliable enough to not need that automated safeguard.

if you have a build that gets stuck on “Waiting for a runner to pick up this job...”, it may be a bug or a github infra failure. you can cancel the build yourself, but please take a screenshot and report it first!

Testing:

Signed-off-by: Delan Azabani <dazabani@igalia.com>
@delan delan requested a review from sagudev as a code owner November 25, 2025 06:00
@servo-highfive servo-highfive added the S-awaiting-review There is new code that needs to be reviewed. label Nov 25, 2025
@servo-highfive servo-highfive removed the S-awaiting-review There is new code that needs to be reviewed. label Nov 25, 2025
@jschwe
Copy link
Member

jschwe commented Nov 25, 2025

Slightly off-topic:

clogging up our github-hosted runner capacity (limited to 60).

Is it not 40 (I'm not sure where to check / see the number)?

@delan
Copy link
Member Author

delan commented Nov 25, 2025

Is it not 40 (I'm not sure where to check / see the number)?

i think we had it upped from 20 to 60 in 2023, but i can't check anymore (it's in org settings).

@sagudev
Copy link
Member

sagudev commented Nov 25, 2025

Is it not 40 (I'm not sure where to check / see the number)?

i think we had it upped from 20 to 60 in 2023, but i can't check anymore (it's in org settings).

60 github hosted runners

@delan delan added this pull request to the merge queue Nov 25, 2025
@servo-highfive servo-highfive added the S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. label Nov 25, 2025
Merged via the queue into main with commit 795e5b4 Nov 25, 2025
32 checks passed
@delan delan deleted the no-runner-timeouts branch November 25, 2025 12:48
@servo-highfive servo-highfive removed the S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. label Nov 25, 2025
delan added a commit to servo/ci-runners that referenced this pull request Dec 2, 2025
unfortunately neither the new global queueing feature
(servo/servo#40852), nor dropping the runner-timeout jobs
(servo/servo#40876), have improved self-hosted runner utilisation (a
metric of server workload).

that said, the self-hosted runner hit rate is 93%, so not too many jobs
are missing out on self-hosted runners.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants