CrowdStrike has promised to improve its software tests after a faulty content update for Windows systems caused a global IT outage on Friday. The cybersecurity company's mistake resulted in problems for banks, hospitals and airlines as millions of PCs displayed “blue screens of death”.
In a detailed review of the incident published on Wednesday, CrowdStrike said there was a “bug” in a system designed to ensure software updates work properly. The review comes as the company faces a backlash for giving $10 Uber Eats vouchers to the “teammates and partners” who dealt with the crisis.
Crowdstrike said the glitch meant “problematic content data” in a file went undetected. The company said it could prevent the incident from happening again with better software testing and checks, including more scrutiny from developers.
The faulty update crashed 8.5 million Microsoft Windows computers around the world and George Kurtz, Crowdstrike's boss, has apologised for the impact of the outage. But cybersecurity experts told BBC News that the review revealed the firm made “major mistakes”.
“What’s clear from the post mortem is they didn't seem to have the right guardrails in place to prevent this type of incident or to reduce the risk of it occurring,” said cyber-security consultant Daniel Card.
His thoughts were echoed by cybersecurity researcher Kevin Beaumont, who said the key lesson from CrowdStrike's review was that the firm doesn't “test in waves”.
“They just deploy to all customers at once in a so called ‘rapid response update' which was obviously a huge mistake,” he said. But Sam Kirkman from cybersecurity firm NetSPI told the BBC the review showed CrowdStrike “took steps” to prevent the outages.
He said these steps “have likely been effective to prevent incidents on countless occasions prior to last week”. Meanwhile, social media users claiming to be CrowdStrike employees have mocked the company for its decision to hand out $10 Uber Eats vouchers as a thank you for their work.
“I literally wanted to drive my car off a bridge this weekend and they bought me coffee. Nice,” said one Reddit user.
“Biggest IT outage ever, here's $10, go buy some coffee or something. Absolute clownshow. This is worse than doing nothing,” another added. CrowdStrike told the BBC it sent these vouchers “to our teammates and partners who have been helping customers through this situation”.
Congress calls
According to insurance firm Parametrix, the top 500 US companies by revenue, excluding Microsoft, had faced some $5.4bn (£4.1bn) in financial losses from the outage. It said that only $540m (£418m) to $1.08bn (£840m) of these losses were insured.
And the US government has opened an investigation into Delta Airlines' handling of the outage after it continued to cancel hundreds of flights.
Delta chief executive Ed Bastian said in a letter to customers on Wednesday that “the worst impacts of the CrowdStrike-caused outage are clearly behind us” and it expects the airline to make a full recovery on Thursday.
Meanwhile, Mr Kurtz has been called to testify in front of Congress about the outage.
“This incident must serve as a broader warning about the national security risks associated with network dependency,” wrote the House Committee on Homeland Security.
It has given the cybersecurity company until Wednesday evening to respond by scheduling a hearing.
— CutC by bbc.com
