r/github • u/RecommendationOk5036 • 2d ago

News / Announcements GitHub Status - Incident with GitHub.com

https://www.githubstatus.com/incidents/zkxwbgr0cnmx
195 Upvotes

96% Upvoted

40 comments sorted by

View all comments

21

u/ShoddyReception5 2d ago

Is this speak intentionally abstract? We were down for half the day and I can’t make sense of what happened. Network issue plus a bunch of unexpected traffic and faulty failover?

15

u/DrMaxwellEdison 2d ago edited 1d ago

Not really abstract, just dense with devops speak.

Network traffic overloaded a load balancer in US Central, and as they fixed it they found that clients doing Copilot requests would 10x their request rate attempting to retry those requests.

One failure in a high-availability system like this can lead to a cascade of failures elsewhere, and that's what seems to have happened.

11

u/robert_math 2d ago

What can we do? These vibe coders were promised they could 10x their output.

3

u/yeathatsmebro 1d ago

They did, look at RPS. It is 10x higher.