Microsoft released several patches in last weeks January 2022 Patch Tuesday that fixed some serious vulnerabilities. You can look at the entire list of updates here.
The Mirazon Group has seen some of these updates cause severe outages caused by domain controllers being stuck in repetitive boot loops, Hyper-V failing to start, and inaccessible ReFS volumes. These issues continue until the updates are removed / rolled back.
Windows Domain Controller Boot Loops
After having installed these updates, Windows domain controllers enter an endless boot loop every few minutes after reaching the login screen. This appears to be related to the LSASS.exe process; the process consumes the CPU and eventually terminates, resulting in a reboot.
The LSASS process is critical to the Windows Operating System to operate and thus will reboot when this process crashes.
If you manage to grab event viewer logs quick enough or have them centrally stored, you will find the following:
The process wininit.exe has initiated the restart of computer [computer_name] on behalf of user for the following reason: No title for this reason could be found Reason Code: 0x50006 Shutdown Type: restart Comment: The system process ‘C:\WINDOWS\system32\lsass.exe’ terminated unexpectedly with status code -1073741819. The system will now shut down and restart.
Hyper-V cannot Start Guest VMs
This is a big one. Once a Hyper-V host receives these updates and reboots, the host will no longer be able to start the Hyper-V Services. Therefore, no guest VMs will run.
When attempting to manually start a guest VM, you will receive the following error:
Virtual machine xxx could not be started because the hypervisor is not running.
ReFS becomes inaccessible after updates
We have seen multiple Windows Resilient File System (ReFS) volumes that are no longer accessible, or are seen as RAW (unformatted) after installing these updates.
Go figure, the resilient file system is brought down by it’s creator. NTFS attached volumes appear to be unaffected by these updates.
Out of Band Updates to Fix
On January 18th, Microsoft released OOB updates in an attempt to resolve these wide spread issues.
All OOB updates can be downloaded on the Microsoft Update Catalog here. Some of them can also be installed directly through Windows Update as optional updates.
The following updates can only be downloaded and installed via the Microsoft Update Catalog:
-
- Windows 8.1, Windows Server 2012 R2: [KB5010794]
- Windows Server 2012: [KB5010797]
Updates for these Windows versions are also available through Windows Update as an optional update:
-
- Windows 11, version 21H1 (original release): [KB5010795]
- Windows Server 2022: [KB5010796]
- Windows 10, version 21H2: [KB5010793]
- Windows 10, version 21H1: [KB5010793]
- Windows 10, version 20H2, Windows Server, version 20H2: [KB5010793]
- Windows 10, version 20H1, Windows Server, version 20H1: [KB5010793]
- Windows 10, version 1909, Windows Server, version 1909: [KB5010792]
- Windows Server 2019: [KB5010791] (Released on 1/18/22)
- Windows 10, version 1607, Windows Server 2016: [KB5010790]
- Windows 10, version 1507: [KB5010789]
- Windows 7 SP1: [KB5010798]
- Windows Server 2008 SP2: [KB5010799]
Uninstall Updates to Fix
The only other option is to entirely remove the Windows Updates and reboot the host. Due to the way that Microsoft bundles all security patches into a single rollup, removing the cumulative updates to fix the aforementioned issues will also remove the fixes that Microsoft implemented.
Run the following commands within an elevated Command Prompt to manually remove these updates:
- Windows Server 2012 R2:
wusa /uninstall /kb:5009624
- Windows Server 2012 R2:
wusa /uninstall /kb:5009595
- Windows Server 2012:
wusa /uninstall /kb:5008604
- Windows Server 2019:
wusa /uninstall /kb:5009557
- Windows Server 2022:
wusa /uninstall /kb:5009555
These commands can be run from Safe Mode as well.
If, however, you are not able to get the host stable enough to issue a Command Prompt, we have been successful with booting to installation media and accessing Command Prompt via the Recovery Console.
The commands are different within this environment and level DISM to achieve the same result.
You will have to used the following command in order to pull the package name:
dism /online /get-packages
It may be useful to pipe this to the clipboard and paste into Notepad:
dism /online /get-packages | clip
Once you have the package name, you can construct an uninstall command as such:
dism /online /Remove-Package /PackageName:Package_for_KB5008604~31bf3856ad364e35~amd64~~9200.23518.1.0