Last month, on July 19, a bug within a routine update of the cybersecurity provider, CrowdStrike, caused arguably the largest IT outage in history. The bug caused widespread disruption across multiple key sectors such as finance, health, and infrastructure. Sound familiar? Yes, Y2K’s fears have come to life 24 years late and reclothed in shiny Silicon Valley overalls.
In 1999, the world held its breath as people prepared themselves for a potential collapse of the globe’s computer systems. Financial databases were going to be deleted, state secrets to be released, and planes to fall out of the sky.
This was all based on the fear that because programmers had lacked the foresight to use a 4-digit code to designate the year and instead used a 2-digit code, computer systems across the globe were going to malfunction, confused that it did not know what the morning after the day before actually was.
Admittedly, for banks, this was a legitimate concern. What if depositors were unable to withdraw funds or complete vital transactions? What if their computer systems would miscalculate interest?
Luckily, Y2K was avoided. In late 1998, the US government passed the Year 2000 Information and Readiness Disclosure Act, which was designed to encourage companies to disclose information about the status of their Year 2000 compliance efforts.
Business and government organizations were so concerned that they created special technology teams to ensure that all hardware and software were Y2K compliant. According to research firm Gartner, businesses and governments spent from $300-$600 billion on Y2K remedies.
If only CrowdStrike had been as cautious.
Whilst Y2K was averted, July’s CrowdStrike update and the global chaos and carnage that ensued appears to have resurrected and realized the fears that the 1999 analogue world so desperately avoided twenty-five years ago.
CrowdStrike’s bug caused widespread disruption: 5,000 flights across the globe were cancelled, banking was stalled, healthcare services ground to a halt, broadcasters dragged off-air, all while facing the now-infamous blue screen of death indicating that systems had fallen victim to this pandemic of disruption.
So, what caused this exactly? What exhumed the fears of Y2K and dumped them into present-day reality?
CrowdStrike, headquartered in Austin, Texas, is one of the most trusted and widely used cybersecurity firms in the world. It offers a cloud-based security platform to prevent malicious attacks on computer systems and the data they hold.
In the past, it has been charged with investigating some of the largest hacks, like the 2014 Sony Pictures hack that leaked personal information of employees and erased Sony’s computer infrastructure.
Last month, on July 19, problems began to emerge for businesses and their computing systems, but it was not clear what caused them. Microsoft appeared to be at the centre of the problem. However, it soon emerged that all the systems facing problems were Microsoft systems with CrowdStrike’s Falcon software installed.
Falcon is CrowdStrike’s flagship product. It works by providing a comprehensive suite of cybersecurity tools that continuously monitor and protect various parts of your computing systems from malware and other threats.
However, this requires constant updates to ensure that it’s always ready to respond and that it always knows where to look, and it needs total access to a system to allow it to search for malware in the most sensitive parts of an operating system.
Cybersecurity and AI expert, Rotem Farkash, said that the Falcon product ‘is certainly unique in the sense that few other products have such access to the most sensitive core components of an operating system.’
While the specific problem has been detailed by CrowdStrike, they shared their Channel File 291 incident Root Cause Analysis (RCA) report on August 6. The summary provides details about the process enhancements and mitigation steps CrowdStrike has taken and will take moving forward. The full RCA and executive summary are available on the CrowdStrike Remediation hub. All of these materials provide a detailed overview of what happened and how CrowdStrike approached remediation.
Since the outage on July 19, CrowdStrike has been able to issue advice and assistance to its customers, but the speed of solutions is reported to vary, with some able to be fixed in a few hours, while others “could be a bit longer,” according to CrowdStrike boss, George Kurtz. IT departments from across the globe now face the painstaking task of having to manually reboot each specific system separately and in safe mode.
Whilst Farkash admitted that ‘CrowdStrike is still an industry leader in terms of its pure protection ability, this recent episode has uncovered that it needs to be careful not to become its own biggest enemy, and whilst these things happen,’ Farkash continued, ‘the fact that this bug was passed through the internal peer review, is profoundly concerning.’
The coming weeks and months will reveal just how much damage this incident has done to the tech giant. However, in the meantime, sit back and bask in some Y2K nostalgia, because now you can say to your friends in their 50s: ‘Y2K? That was nothing compared to Y2K24.’