[Guide] Headless Recovery: Fixing Corrupted NetworkManager Wi-Fi State After a Power Cut

A sudden power loss on a headless Raspberry Pi (Pi Zero 2 W, Pi 3, Pi 4, or Pi 5) running Raspberry Pi OS (Bookworm) can leave the system inaccessible over Wi-Fi. If the interruption occurs while NetworkManager is updating its state or configuration files, the network service may fail to initialize correctly, preventing remote access. Rather than following the common recommendation of re-flashing the SD card—which can result in unnecessary data loss—this guide demonstrates an out-of-band recovery method that preserves the existing installation.

The recovery process begins by mounting the Raspberry Pi's SD card on a secondary Linux machine and entering its filesystem through a chroot environment. However, this introduces a classic Catch-22: repairing the network configuration requires executing the Raspberry Pi's ARM binaries, which cannot run natively on a typical x86-64 Linux system.

To overcome this limitation, we use QEMU user-mode emulation to execute the Pi's ARM utilities while the SD card remains offline. An emergency fallback hotspot profile is then added to NetworkManager's configuration so that, on the next boot, the Raspberry Pi automatically starts a recovery hotspot (Pi-Rescue). This temporary wireless network restores SSH access, allowing the underlying networking issue to be repaired without re-imaging the SD card.

Step-by-Step Technical Recovery Guide

Important Note for Intel/AMD PC Users :

If you are running this recovery process from a standard PC (x86_64) rather than another Raspberry Pi (ARM), your computer cannot natively run the Pi's binaries. Before proceeding you can check your machine's architecture by running:

   uname -m

If the command returns x86_64, you must install QEMU user emulation on your host Linux machine before proceeding:

   sudo apt update && sudo apt install qemu-user-static binfmt-support

Step 1: Filesystem Repair & System chroot

NetworkManager configurations cannot easily be parsed or prioritized before the system boots if they are dropped in as raw text profiles. We must mount the SD card on a secondary Linux machine and use chroot (change root) to execute native configuration commands inside the Pi's environment directly from our laptop.

Find your SD card's identifier (e.g., look for the main rootfs partition, which might show up as /dev/sdb2):
```
lsblk
```
Repair any broken filesystem structures caused by the abrupt shutdown:
```
sudo fsck -y /dev/sdb2
```

Mount the partition and bind vital system virtual directories to prepare for the chroot environment:

sudo mkdir -p /mnt/pi
sudo mount /dev/sdb2 /mnt/pi
sudo mount --bind /dev /mnt/pi/dev
sudo mount --bind /sys /mnt/pi/sys
sudo mount --bind /proc /mnt/pi/proc

chroot into the Raspberry Pi's environment to execute commands as the root user:
```
sudo chroot /mnt/pi
```

Step 2: Create the Emergency Hotspot & Configure Startup Priorities

Now that your terminal session is running inside the Pi's filesystem context, use the native nmcli binary to build a fallback network profile that absolute precedence on startup.

Generate the native recovery hotspot profile:

nmcli device wifi hotspot ifname wlan0 ssid "Pi-Rescue" password "your_password"

Instruct NetworkManager that this connection profile is explicitly allowed to autoconnect on system boot:
```
nmcli connection modify "Pi-Rescue" connection.autoconnect yes
```
Boost the autoconnect priority to high (90). This forces the Pi to launch this network interface profile first, overriding any standard client Wi-Fi configurations during boot initialization:
```
nmcli connection modify "Pi-Rescue" connection.autoconnect-priority 90
```
Verification Tip: Since the NetworkManager background service isn't actively running for the SD card, standard commands (nmcli connection show) won't display the changes while you are in chroot. If you want to verify that your priority was written successfully, open a separate terminal window on your laptop and check the raw configuration file directly:
```
sudo cat /mnt/pi/etc/NetworkManager/system-connections/Pi-Rescue.nmconnection
```
Exit the active chroot environment:
```
exit
```
Cleanly unmount the system tree on your host laptop before pulling out the SD card:
```
sudo umount -R /mnt/pi
```

Step 3: Boot, SSH Login, and State Repair

Unplug the SD card from your host laptop, insert it back into the Raspberry Pi, and power it on. Thanks to the priority variables we injected, the Pi will immediately begin broadcasting the Pi-Rescue network.

Connect your laptop or workstation directly to the Pi-Rescue WiFi access point.
Open a terminal on your laptop and ssh into the Pi using NetworkManager's standard default hotspot gateway IP (10.42.0.1):
```
ssh your_username@10.42.0.1
```
To verify the IP address of Pi's recovery hotspot use:
```
ip addr show wlan0
```
The Core Fix: Once inside the terminal, safely delete the corrupted network state file that was damaged during the power cut:
```
sudo rm -f /var/lib/NetworkManager/NetworkManager.state
```
Force a physical wireless environment rescan to wake up the network interfaces and verify your main WiFi(e.g., your home router or phone hotspot) is visible:
```
sudo nmcli device wifi rescan
sudo nmcli device wifi list
```

Re-authenticate and establish a fresh connection to your primary network:

sudo nmcli device wifi connect "your_router's_ssid" password  "your_wifi's_password"

Ensure your primary client network has a maximum priority (e.g., 100):

sudo nmcli connection modify "your_router's_ssid" connection.autoconnect-priority 100

Check whether your change was applied successfully:

sudo nmcli connection show "your_router's_ssid" | grep autoconnect-priority

Step 4:Cleanup(Remove the Backup Hotspot from Startup)

To prevent the Pi from defaulting back into a localized recovery loop on every subsequent reboot, the emergency profile must be adjusted or removed from active rotation.

Option A(Recommended): Drop the emergency profile's autoconnect priority to a negative value. This leaves the configuration intact on the disk as a silent fallback that will only spin up if your primary network disappears completely:
```
sudo nmcli connection modify "Pi-Rescue" connection.autoconnect-priority -10
```

Check all the networks ranked by priority:

sudo nmcli -f NAME,AUTOCONNECT-PRIORITY connection show

Option B: Completely delete the emergency recovery profile from the storage configuration:
```
sudo nmcli connection delete "Pi-Rescue"
```

Finally,restart NetworkManager and ensure your Pi successfully jumps back onto your main WiFi network:

sudo systemctl restart NetworkManager

This guide is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) License. You are free to share and adapt this content as long as you give appropriate credit to the original author (Kernovax).

Kernovax/headless-recovery.md

Select an option

No results found