Resolved -
On May 15, 2026, between 19:13 UTC and 19:34 UTC, users in the Central US and Sweden Central GHEC DR regions experienced failures when pushing to Git repositories. On average, Git HTTP Push Operations saw 10.9% errors, with a peak minute of 31.5%. Actions saw 10% rate of run start delays >5m a 5% failure rate in queueing new runs. The issue was caused by a configuration change that inadvertently stopped background jobs across multiple regions. When the fix was deployed and background jobs restarted, the backlog of queued jobs caused a thundering herd to backend systems, resulting in push failures. The team mitigated the issue by throttling the job queues, and all services recovered by 19:34 UTC. We are implementing queue depth monitoring and alerts for background job processing, as well as improving testing coverage to prevent similar issues.
May 15, 19:53 UTC
Update -
Impact has been mitigated in the Central US and Sweden Central regions. The underlying issue was correlated to background jobs which had stopped running earlier. When they were started again, repository storage hosts experienced high load, failing some pushes.
All background jobs have completed at this time and we've observed full recovery.
May 15, 19:37 UTC
Monitoring -
The degradation has been mitigated. We are monitoring to ensure stability.
May 15, 19:34 UTC
Update -
We are currently investigating increased Git client push operations in the GHEC DR environments in the Central US and Sweden Central regions.
May 15, 19:21 UTC
Update -
The degradation affecting Git Operations has been mitigated. We are monitoring to ensure stability.
May 15, 19:20 UTC
Investigating -
We are investigating reports of degraded performance for Git Operations
May 15, 19:18 UTC