After a system update pushed by CrowdStrike on Friday upended government services, emergency call centers, banks, airlines, hospitals, and other businesses, most systems are up and running as normal now.
CrowdStrike is actively assisting customers affected by a defect in a recent content update for Windows hosts. Mac and Linux hosts were not impacted. The issue has been identified and isolated, and a fix has been deployed. This was not a cyberattack, the company stressed.
Customers are advised to check the support portal for updates.
“We assure our customers that CrowdStrike is operating normally, and this issue does not affect our Falcon platform systems. If your systems are operating normally, there is no impact to their protection if the Falcon sensor is installed,” the company said in a statement.
Addditonally, Fenix24 issued an open-source solution for global CrowdStrike-related issues. The Fenix24 scripts provide rapid remediation at scale and include solutions for computers and virtual machines already affected by the "blue screen of death" message.
The fix previously released by CrowdStrike on Friday resolved the issue for computers not yet in the blue screen mode. Fenix24's scripts were created for Windows and VMware using public information and the Fenix24 team's internal expertise.
The Windows scripts force the reboot of machines into Safe Mode and then remove the problematic file. These Windows scripts are executed leveraging a provided GPO.
However, if the drive is secured with Bitlocker, users will need to enter the Bitlocker key manually and then proceed to Safe Mode. The VMware scripts leverage a working server to detach the virtual disk, mount it, remove the problematic file, dismount it, reattach it to the problem VM, and then reboot it.
The scripts are free of charge and publicly available as a part of Fenix24's mission to be “Good Samaritans,” providing help to the broader community with the goal of achieving a more secure world, according to the company.
Access the scripts here
"We woke up today and had dozens of requests for Fenix24 to come onsite to help remediate this technical issue causing global outages. Instead, we decided to use the full force of the Fenix24 and Conversant Battalions to develop a scalable remediation solution in real-time to help everyone solve this problem and publish it for free," said Heath Renfrow, Fenix24 co-founder. "That's who we are—the fastest and most efficient remediation firm on the planet with an arm outstretched to help those in need."
While this was not a security breach, the resulting disruption created an opportunity for malicious actors, Keatron Evans, VP portfolio and product strategy, instructor and author at Infosec, warned. “Cybercriminals often exploit such vulnerabilities, and with resources diverted to address this issue, organizations may be more susceptible to attacks. It's crucial to remain vigilant during this period,” he said. “This incident highlights the reality of software supply chain risk. There will be questions about how a security product could cause such extensive disruption and hold so much control over computing devices. The answer lies in the necessity for deep integration and elevated privileges. To effectively protect against threats, security products like CrowdStrike require these permissions to detect and neutralize malware that may have compromised elevated credentials. Without these privileges, the security software would struggle to combat high-level threats.”
While it appears the global outage wasn’t due to a cyberattack this time, this incident is a stark reminder that organizations need to be proactive when it comes to protecting their critical assets and planning for business continuity following a disruption, said Danielle Sheer, chief trust officer of Commvault.
“Being offline for any period of time creates chaos, costs money, and can damage a company’s reputation and trust. Organizations that regularly test and retest their recovery capabilities—including measuring their security posture and evaluating their backup systems—are best positioned to rapidly and thoroughly recover after an incident,” Sheer said.