Introduction
On Friday, a faulty update from CrowdStrike caused an estimated 8.5 million Windows devices to crash. This update led to widespread Blue Screen of Death (BSOD) errors and reboot loops, affecting numerous organizations worldwide. To mitigate this issue, Microsoft has released a custom WinPE recovery tool.
The Incident
What Happened?
The problematic update from CrowdStrike resulted in millions of Windows devices experiencing BSOD errors and entering continuous reboot loops. This caused significant IT outages, disrupting operations in airports, hospitals, banks, companies, and government agencies globally.
The Impact
With all affected Windows devices rendered non-functional, organizations faced severe operational disruptions. Admins were tasked with rebooting impacted devices into Safe Mode or the Recovery Environment to manually remove the buggy kernel driver from the C:\Windows\System32\drivers\CrowdStrike folder. Given the sheer number of devices, this manual process proved to be highly problematic, time-consuming, and difficult.
Microsoft’s Solution
Introducing the Custom Recovery Tool
To assist IT admins and support staff, Microsoft has developed a custom recovery tool that automates the removal of the faulty CrowdStrike update, allowing affected devices to boot normally again.
“As a follow-up to the CrowdStrike Falcon agent issue impacting Windows clients and servers, we have released a USB tool to help IT Admins expedite the repair process,” reads a Microsoft support bulletin.
How to Access the Tool
The signed Microsoft Recovery Tool can be downloaded from the Microsoft Download Center.
Using the Recovery Tool
Requirements
To use Microsoft’s recovery tool, IT staff need:
- A Windows 64-bit client with at least 8 GB of space
- Administrative privileges on the device
- A USB drive with at least 1 GB of storage
- A Bitlocker recovery key, if necessary
Note: A USB flash drive of 32GB or smaller is needed to format it with FAT32, which is required for booting.
Creating the Recovery Tool
The recovery tool is created via a PowerShell script from Microsoft, which needs to be run with administrative privileges. The script formats a USB drive, creates a custom WinPE image, copies it to the drive, and makes it bootable.
Running the Tool
Once the USB key is ready, boot the impacted Windows device using it. The tool will automatically run a batch file named CSRemediationScript.bat.
- The batch file prompts for any necessary Bitlocker recovery keys.
- It searches for the faulty CrowdStrike kernel driver in the C:\Windows\system32\drivers\CrowdStrike folder.
- If the driver is found, it is automatically deleted.
Completing the Process
BleepingComputer’s review indicates that the batch file does not create logs or backups of the CrowdStrike driver. After completing its task, the script prompts you to press any key to reboot the device. Once the CrowdStrike driver is deleted, the device should boot back into Windows and become operational again.
Challenges and Considerations
Retrieving Bitlocker Recovery Keys
Windows admins’ primary challenge is retrieving any required Bitlocker recovery keys. Therefore, identifying and recovering these keys should be the first step before attempting to recover the devices.
Conclusion
Microsoft’s custom WinPE recovery tool provides a streamlined solution for resolving the faulty CrowdStrike update issue. By automating the removal process, it saves time and effort for IT admins, allowing organizations to restore normal operations more efficiently.