The official causes, for now, of both the Microsoft and FAA outages last week have turned out to be essentially the same thing: a self-inflicted DDoS.
A DDoS (Distributed Denial of Service) is when a huge number of computers flood a service with so much traffic that it gets overwhelmed and stops working. Usually done for nefarious reasons by third-parties but, sometimes, like here, people end up doing it to themselves, which takes a special kind of talent.
A few months ago, Microsoft unknowingly introduced a change to their Microsoft Gateway product that caused it to fall apart under high load. Last week, due to some other servers failing, the traffic got redirected and crashed Microsoft Gateway. Crash, loop, repeat, ad infinitum.
Most of the FAA network still runs over nineties T1 copper lines, not today’s high-bandwidth fibre. Their newly deployed SMART AI seems to have caused switches to flood the network and stop all traffic – in the cables and, less amusingly, in the sky.
You could say bugs happen, these were bugs, lessons learned, case closed. But if you dig beyond the bugs themselves, the real cause will very often be something else.
Which brings me to yesterday’s bogeymen: AI and layoffs.
Every time some big service goes down, the people involved make sure to announce loudly and in unison that neither was the cause and to please stop the scaremongering. And, in a very narrow sense, they’re probably right. Nobody’s saying a chatbot “went rogue” and took down Microsoft.
The direct cause was a bug or overloading, but “direct” is doing a lot of work here. It’s a handy way to terminate the conversation.
Think of hurricanes and climate change. You can’t blame climate change for a specific storm, but you can watch storms like it get more and more frequent.
Same thing here. You can’t pin a specific outage on a round of layoffs or a pile of AI-authored code. But you can watch failure rates increase when the conditions for them become normalised: people with deep system knowledge and salaries to match being shown the door, maintenance and reliability work that never gets done because it’s never rewarded (no one ever got promoted for the outage that didn’t happen), and finally, generative AI code coming in faster than anyone can realistically understand or review.
None of this is a technical problem, it’s a human one. Generative AI is an amplifier, an accelerant. A company that only cares about its stock price and holds its engineers in contempt now has a tool for cutting corners at unprecedented speed and a handy story for laying off the people who might have been the ones to object.
The rot was already there, and the AI-enabled fantasy that expertise is optional gave management permission to stop pretending.
Expect more of these events, and more often, for as long as these conditions persist.
So now would be a good time to make sure you’ve checked the fire exits and have a backup plan.
Colin