Echoes of the July 19 CrowdStrike glitch are likely to reverberate across the industry for years to come. For now, IT teams remain focused on slogging through a labor-intensive recovery.
But recovery is just the beginning. What's sure to follow is a barrage of regulatory oversight, hard feelings among the IT community, and a tough reminder that even a small slip-up in a software update can have catastrophic global consequences.
Cyber adversaries have also started to circle, eyeing an opportunity.
The faulty sensory configuration update to the Falcon Platform was released on July 19 at 4:09 UTC, according to CrowdStrike. Once the CrowdStrike update was pushed out, it triggered widespread Microsoft outages across CrowdStrike's 29,000 customers who rely on the company's software for cybersecurity endpoint detection and response (EDR). CrowdStrike's customers include retailers Target and Amazon, tech giants Alphabet and Intel, as well as many other household company names. When they tried to log on Friday morning, employees at some of the world's largest organizations were left staring at the dreaded blue screen of death. Airports, banks, hospitals, governments — there were few sectors spared the fallout — paralyzing the world's economy and causing panic.
It wasn't a cyberattack, CrowdStrike assured the world, just a glitch. But that was little comfort to IT teams who faced Friday with the task of manually booting affected PCs into recovery mode, deleting the bad file, and restarting. That process is still underway in many organizations.
"This is not something that can be done remotely, and in many organizations, will require an administrator," said Tom Marsland, vice president of technology for Cloud Range, in a statement. "This means someone from IT support going computer to computer and doing this manually."
Marsland predicted the recovery will take days, even a week or more, for some larger companies.
"Recovery is going to be painful, to put it lightly," Marsland added.
Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.