Skip to content

Instantly share code, notes, and snippets.

@jlmalone

jlmalone/exfat-crash-recovery-macos.md

Created February 7, 2026 06:23
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jlmalone/a3ef3260d55f8a86c567e6c34f4d80f2 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jlmalone/a3ef3260d55f8a86c567e6c34f4d80f2 to your computer and use it in GitHub Desktop.
Download ZIP
ExFAT crash recovery on macOS — fix allocation bitmap corruption after kernel panics/power loss
Raw
exfat-crash-recovery-macos.md

ExFAT Crash Recovery on macOS

After a macOS crash (kernel panic, power loss, forced restart), external ExFAT drives often become unmountable. macOS detects filesystem corruption but cannot repair it, leaving you stuck in a loop where every mount attempt fails.

This guide explains what's happening at the filesystem level and how to fix it — from a quick read-only mount to a full bitmap rebuild that restores read-write access.

Table of Contents

The Problem

When macOS crashes while an ExFAT volume is mounted, two things typically go wrong:

  1. The dirty flag stays set. ExFAT sets VolumeFlags bit 1 (VolumeDirty) when a volume is mounted for writing. A clean unmount clears it. A crash leaves it set, telling the OS "this filesystem may be inconsistent."

  2. The allocation bitmap becomes corrupt. The allocation bitmap tracks which clusters are in use. During normal operation, macOS updates the bitmap in memory and flushes it to disk periodically. A crash can leave the on-disk bitmap out of sync with the actual FAT (File Allocation Table), which is the authoritative record of cluster chains.

The result: fsck_exfat detects the bitmap inconsistency, reports "The bitmap needs to be repaired", but then fails to repair it and exits with code 1.

Since diskutil mount runs fsck_exfat before mounting, and fsck fails, the mount is refused:

$ diskutil repairVolume diskXsY
Started file system repair on diskXsY
Performing fsck_exfat -y -x /dev/rdiskXsY
Checking volume
Checking main boot region
Checking system files
Checking upper case translation table
Checking file system hierarchy
Checking active bitmap
The bitmap needs to be repaired
Rechecking main boot region
Rechecking alternate boot region
File system check exit code is 1
Restoring the original state found as unmounted
Error: -69845: File system verify or repair failed
Underlying error: 1

Your drive is fine. Your data is fine. macOS just can't get past the bitmap check.

Why macOS Makes This Hard

There are several layers of macOS that conspire to make this recovery harder than it needs to be.

1. diskarbitrationd auto-spawns fsck_exfat

When macOS detects an ExFAT volume, diskarbitrationd automatically launches fsck_exfat to check it. If you kill the fsck process, diskarbitrationd just starts a new one. On a large drive (e.g., 4 TB), each fsck run takes minutes to scan the full filesystem hierarchy before getting to the bitmap check and failing.

# This just makes things worse — it restarts from scratch
$ sudo kill <fsck_exfat_pid>
# diskarbitrationd immediately spawns a new one

2. fsck_exfat detects but cannot repair bitmap corruption

Apple's fsck_exfat can identify that the bitmap is inconsistent with the FAT, but its repair capability is limited. For large-scale bitmap corruption (common after crashes), it simply fails. There is no -f force-repair flag. There is no alternative fsck for ExFAT on macOS — fsck.exfat and exfatfsck (from the Linux exfatprogs package) are not available.

3. diskutil mount requires fsck to pass

You cannot tell diskutil mount to skip the filesystem check. It will always run fsck_exfat first, and if fsck fails, it refuses to mount:

Volume on diskXsY failed to mount
If you think the volume is supported but damaged, try the "readOnly" option

Even diskutil mount readOnly still runs fsck first.

4. SIP blocks raw device access from non-Apple binaries

System Integrity Protection (SIP) restricts which processes can perform raw I/O on disk devices. Python, dd run from non-entitled contexts, and other unsigned tools get Operation not permitted when trying to open /dev/rdiskXsY for writing:

$ sudo python3 fix_bitmap.py /dev/rdiskXsY
OSError: [Errno 1] Operation not permitted

Even with sudo, SIP blocks the operation because the binary isn't Apple-signed with the appropriate entitlements.

The workaround: Run commands from Terminal.app (or iTerm, etc.) with Full Disk Access enabled in System Preferences → Privacy & Security → Full Disk Access. The terminal's TCC entitlement is inherited by child processes, including sudo python3 and sudo dd.

5. Python's direct write() to raw devices can fail

Even with proper entitlements, Python's file.write() to a raw device (/dev/rdiskXsY) may fail with OSError: [Errno 5] Input/output error or [Errno 22] Invalid argument. This is because:

  • Writes must be aligned to the device's sector size (512 bytes)
  • The write size must be a multiple of the sector size
  • Some USB-to-SATA bridges are picky about write patterns

The fix: Write to a temp file, then use dd (an Apple-signed binary) to copy it to the device. dd handles alignment and is entitled for raw device I/O.

Quick Fix: Read-Only Mount

If you just need to read your files (copy them off the drive), you can bypass fsck entirely using mount_exfat directly:

# 1. Force-unmount to stop any running fsck
sudo diskutil unmountDisk force diskX

# 2. Create a mount point
sudo mkdir -p /Volumes/MyDrive

# 3. Mount read-only, bypassing fsck
sudo mount_exfat -o rdonly /dev/diskXsY /Volumes/MyDrive

This works because mount_exfat doesn't run fsck — it just loads the exfat.kext kernel extension and mounts the volume. The bitmap only matters for write operations (allocating new clusters), so read-only mode is perfectly safe even with a corrupt bitmap.

Once mounted, copy your files off, then proceed to the full fix if you want read-write access restored.

Full Fix: Rebuild the Allocation Bitmap

The allocation bitmap can be deterministically rebuilt from the FAT. The FAT is the authoritative record of which clusters belong to which file chains. If a FAT entry is non-zero, the cluster is allocated; if zero, it's free. We just need to walk the FAT and set the corresponding bitmap bits.

Prerequisites

  • macOS Terminal with Full Disk Access (System Preferences → Privacy & Security → Full Disk Access)
  • Python 3 (ships with macOS or Xcode Command Line Tools)
  • The drive must be unmounted (we'll force-unmount it)

Using the Script

The companion script fix-exfat-macos.sh automates the entire process:

# Diagnose only (safe, no writes)
sudo bash fix-exfat-macos.sh --dry-run /dev/diskXsY

# Quick read-only mount
sudo bash fix-exfat-macos.sh --read-only /dev/diskXsY

# Full repair (rebuilds bitmap, clears dirty flag, mounts read-write)
sudo bash fix-exfat-macos.sh /dev/diskXsY

What the Script Does

  1. Reads the boot sector — parses sector size, cluster size, FAT location, cluster heap offset, cluster count
  2. Checks the dirty flag — reads VolumeFlags at offset 0x6A in the boot sector
  3. Reads the entire FAT — the FAT maps every cluster to its next cluster in a chain (or marks it as free/end-of-chain)
  4. Rebuilds the bitmap from the FAT — for each cluster: if the FAT entry is non-zero, set the corresponding bit in the bitmap
  5. Writes the new bitmap to disk — saves to a temp file, then uses dd to write it to the bitmap's on-disk location
  6. Clears the dirty flag — updates VolumeFlags and recalculates the VBR checksum
  7. Mounts the volume — tries diskutil mount first; falls back to mount_exfat if fsck still fails
  8. Disables Spotlight — prevents macOS from immediately hammering the freshly repaired volume with indexing

Manual Steps (If You Prefer)

If you want to understand each step rather than running the script:

# Step 1: Force unmount
sudo diskutil unmountDisk force diskX

# Step 2: Run the bitmap rebuild (embedded in the script, or use the Python directly)
sudo python3 rebuild_bitmap.py /dev/rdiskXsY

# Step 3: Try mounting
sudo diskutil mount diskXsY

# Step 4: If diskutil mount fails (fsck still unhappy), mount directly
sudo mkdir -p /Volumes/MyDrive
sudo mount_exfat /dev/diskXsY /Volumes/MyDrive

# Step 5: Disable Spotlight indexing
sudo mdutil -i off /Volumes/MyDrive

ExFAT On-Disk Structure Primer

Understanding the on-disk layout helps you reason about what the repair script is doing and why.

Overall Layout

Offset (sectors)   Region
─────────────────  ────────────────────────────
0                  Main Boot Sector (512 bytes)
1-8                Extended Boot Sectors
9-10               OEM Parameters + Reserved
11                 VBR Checksum Sector
12-23              Backup Boot Region (copy of 0-11)
24+                FAT Region
24+FAT_LENGTH      Cluster Heap (data area)

The exact offsets depend on the volume — they're stored in the boot sector.

Boot Sector (Sector 0)

Key fields for recovery:

Offset Size Field Notes
0x03 8 FileSystemName Must be EXFAT
0x40 8 PartitionOffset Sectors from disk start
0x48 8 VolumeLength Total sectors
0x50 4 FatOffset FAT start (sectors from volume start)
0x54 4 FatLength FAT size in sectors
0x58 4 ClusterHeapOffset Data area start (sectors)
0x5C 4 ClusterCount Total data clusters
0x60 4 FirstClusterOfRootDirectory Root dir cluster index
0x6A 2 VolumeFlags Bit 0 = ActiveFAT, Bit 1 = VolumeDirty
0x6C 1 BytesPerSectorShift e.g., 9 → 512 bytes
0x6D 1 SectorsPerClusterShift e.g., 9 → 512 sectors/cluster
0x70 1 PercentInUse 0-100, or 0xFF if unknown

FAT (File Allocation Table)

The FAT starts at FatOffset sectors and spans FatLength sectors. Each 4-byte entry corresponds to a cluster (starting at cluster index 2):

Value Meaning
0x00000000 Free cluster
0x00000002 – 0xFFFFFFF6 Next cluster in chain
0xFFFFFFF7 Bad cluster
0xFFFFFFFF End of chain

Cluster 0 and 1 are reserved (media type descriptor and padding).

Allocation Bitmap

The allocation bitmap is a special file stored in the cluster heap. Its location is recorded in an Allocation Bitmap Directory Entry (entry type 0x81) in the root directory. The entry contains:

  • FirstCluster (offset 20): Starting cluster of the bitmap data
  • DataLength (offset 24): Bitmap size in bytes

Each bit corresponds to a cluster (bit 0 of byte 0 = cluster 2, bit 1 = cluster 3, etc.). A 1 bit means allocated; 0 means free.

The bitmap exists so the filesystem can quickly find free clusters without scanning the entire FAT. But the FAT is the authoritative record — if they disagree, the FAT wins, which is why rebuilding the bitmap from the FAT is safe.

VBR Checksum

The first 12 sectors of the volume (boot sector + extended boot sectors + OEM parameters + reserved) are protected by a checksum stored in sector 11. If you modify any of these sectors (like clearing the dirty flag in the boot sector), you must recalculate this checksum or the volume won't mount.

The checksum algorithm (from the ExFAT spec):

def vbr_checksum(sectors_data, bytes_per_sector):
    """Calculate VBR checksum over sectors 0-10."""
    checksum = 0
    for i in range(bytes_per_sector * 11):
        # Skip VolumeFlags (bytes 106-107) and PercentInUse (byte 112)
        if i == 106 or i == 107 or i == 112:
            continue
        checksum = ((checksum << 31) | (checksum >> 1)) + sectors_data[i]
        checksum &= 0xFFFFFFFF
    return checksum

Troubleshooting FAQ

"Operation not permitted" when running the script

Your terminal doesn't have Full Disk Access. Go to System Preferences → Privacy & Security → Full Disk Access and add Terminal.app (or iTerm2, etc.). You may need to restart the terminal.

"Input/output error" during bitmap write

This usually means diskarbitrationd remounted the volume while you were writing. Make sure you:

  1. Force-unmount first: sudo diskutil unmountDisk force diskX
  2. Run the script immediately after unmounting
  3. If it persists, the USB connection may be flaky — try a different port or cable

fsck_exfat keeps running in the background

diskarbitrationd automatically spawns fsck when it detects the volume. You can temporarily suppress auto-mounting:

# Stop disk arbitration (prevents auto-fsck on plug-in)
sudo launchctl unload /System/Library/LaunchDaemons/com.apple.diskarbitrationd.plist

# ... do your work ...

# Re-enable disk arbitration
sudo launchctl load /System/Library/LaunchDaemons/com.apple.diskarbitrationd.plist

Warning: Disabling diskarbitrationd stops all automatic disk mounting system-wide. Re-enable it when done.

"Volume on diskXsY failed to mount" even after bitmap repair

This means diskutil mount ran fsck and fsck still failed. Common reasons:

  • Dirty flag still set — the script should clear it, but verify manually by reading offset 0x6A
  • VBR checksum mismatch — if you cleared the dirty flag, the checksum sector (sector 11) must be updated
  • Other filesystem damage — the bitmap was the main issue, but Spotlight index files (.Spotlight-V100) or other metadata may also be inconsistent

Workaround: Mount directly with mount_exfat (bypasses fsck):

sudo mkdir -p /Volumes/MyDrive
sudo mount_exfat /dev/diskXsY /Volumes/MyDrive

This mounts read-write. If you want to be cautious, add -o rdonly.

"No such file or directory" when mounting

The mount point directory must exist. Create it first:

sudo mkdir -p /Volumes/MyDrive

How do I find my drive's device path?

diskutil list external

Look for the ExFAT partition. It will be something like disk2s1 or disk4s2. Use /dev/diskXsY for mount commands and /dev/rdiskXsY for raw access (dd, Python scripts) — the r prefix gives you the raw (character) device, which is faster for sequential I/O.

Is this safe? Can it make things worse?

The bitmap rebuild is safe because:

  • It only writes the allocation bitmap and boot sector — it never touches the FAT or your actual data
  • The bitmap is derived from the FAT, which is the authoritative record of cluster allocation
  • --dry-run mode lets you see exactly what would change before writing anything

The only risk is from the physical USB connection being unreliable during the write. Use --dry-run first to validate, then run the full repair.

My drive is larger than 2 TB — will this work?

Yes. ExFAT supports volumes up to 128 PB. The script handles large cluster counts and FAT sizes correctly. It was originally developed and tested on a 4 TB drive.

References

This guide was written after recovering a 4 TB external drive that became unmountable after a macOS crash. The bitmap had 1.7 million bytes of differences — fsck_exfat couldn't fix it, but rebuilding from the FAT took under a second.

Raw
fix-exfat-macos.sh
#!/usr/bin/env bash
# fix-exfat-macos.sh — ExFAT crash recovery for macOS
#
# Repairs allocation bitmap corruption caused by crashes/power loss.
# Rebuilds the bitmap from the FAT, clears the dirty flag, recalculates
# the VBR checksum, and mounts the volume.
#
# Usage:
# sudo bash fix-exfat-macos.sh [OPTIONS] /dev/diskXsY
#
# Options:
# --dry-run Diagnose only, don't write anything
# --read-only Just mount read-only (bypasses fsck, no repair)
# --mount-point Path to mount point (default: auto-detected)
# --help Show this help
#
# Requirements:
# - macOS with Full Disk Access for your terminal
# - Python 3 (ships with Xcode Command Line Tools)
# - Must be run as root (sudo)
set -euo pipefail
# ─── Colors & output helpers ─────────────────────────────────────────────────
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
BOLD='\033[1m'
NC='\033[0m'
info() { echo -e "${BLUE}[INFO]${NC} $*"; }
ok() { echo -e "${GREEN}[OK]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
err() { echo -e "${RED}[ERROR]${NC} $*" >&2; }
step() { echo -e "\n${BOLD}=== Step $1: $2 ===${NC}\n"; }
# ─── Argument parsing ────────────────────────────────────────────────────────
DRY_RUN=false
READ_ONLY=false
MOUNT_POINT=""
DEVICE=""
usage() {
sed -n '2,/^$/s/^# \?//p' "$0"
exit 0
}
while [[ $# -gt 0 ]]; do
case "$1" in
--dry-run) DRY_RUN=true; shift ;;
--read-only) READ_ONLY=true; shift ;;
--mount-point) MOUNT_POINT="$2"; shift 2 ;;
--help|-h) usage ;;
/dev/*) DEVICE="$1"; shift ;;
*) err "Unknown argument: $1"; usage ;;
esac
done
if [[ -z "$DEVICE" ]]; then
echo ""
info "No device specified. Scanning for external ExFAT volumes..."
echo ""
# Auto-discover ExFAT partitions on external disks
CANDIDATES=()
while IFS= read -r line; do
CANDIDATES+=("$line")
done < <(diskutil list external 2>/dev/null | grep -i "Microsoft Basic Data" | awk '{print $NF}' || true)
if [[ ${#CANDIDATES[@]} -eq 0 ]]; then
err "No external ExFAT partitions found."
err "Usage: sudo bash $0 /dev/diskXsY"
exit 1
elif [[ ${#CANDIDATES[@]} -eq 1 ]]; then
DEVICE="/dev/${CANDIDATES[0]}"
info "Found one ExFAT partition: ${BOLD}$DEVICE${NC}"
else
info "Found multiple ExFAT partitions:"
for i in "${!CANDIDATES[@]}"; do
VOL_NAME=$(diskutil info "${CANDIDATES[$i]}" 2>/dev/null | grep "Volume Name" | sed 's/.*: *//' || echo "Unknown")
VOL_SIZE=$(diskutil info "${CANDIDATES[$i]}" 2>/dev/null | grep "Disk Size" | sed 's/.*: *//' | cut -d'(' -f1 || echo "Unknown")
echo " [$((i+1))] /dev/${CANDIDATES[$i]} — $VOL_NAME ($VOL_SIZE)"
done
echo ""
read -rp "Select partition [1-${#CANDIDATES[@]}]: " CHOICE
DEVICE="/dev/${CANDIDATES[$((CHOICE-1))]}"
fi
echo ""
fi
# Derive raw device path (rdiskXsY) for direct I/O
RAW_DEVICE="${DEVICE/\/dev\/disk/\/dev\/rdisk}"
# Derive the whole-disk identifier (diskX) for unmounting
WHOLE_DISK=$(echo "$DEVICE" | sed 's|/dev/||; s/s[0-9]*$//')
# ─── Preflight checks ────────────────────────────────────────────────────────
if [[ $EUID -ne 0 ]]; then
err "This script must be run as root (use sudo)"
exit 1
fi
if [[ ! -e "$DEVICE" ]]; then
err "Device $DEVICE does not exist"
exit 1
fi
if ! command -v python3 &>/dev/null; then
err "Python 3 is required but not found"
err "Install Xcode Command Line Tools: xcode-select --install"
exit 1
fi
# ─── Read-only quick mount ───────────────────────────────────────────────────
if [[ "$READ_ONLY" == true ]]; then
step 1 "Read-only mount (bypassing fsck)"
info "Force-unmounting $WHOLE_DISK..."
diskutil unmountDisk force "$WHOLE_DISK" 2>/dev/null || true
# Determine mount point
if [[ -z "$MOUNT_POINT" ]]; then
VOL_NAME=$(diskutil info "$DEVICE" 2>/dev/null | grep "Volume Name" | sed 's/.*: *//' || echo "ExFAT_Recovery")
# Sanitize volume name for use as directory
VOL_NAME=$(echo "$VOL_NAME" | tr -cd '[:alnum:]._-')
[[ -z "$VOL_NAME" ]] && VOL_NAME="ExFAT_Recovery"
MOUNT_POINT="/Volumes/$VOL_NAME"
fi
info "Creating mount point at $MOUNT_POINT..."
mkdir -p "$MOUNT_POINT"
info "Mounting read-only via mount_exfat (no fsck)..."
if mount_exfat -o rdonly "$DEVICE" "$MOUNT_POINT"; then
ok "Volume mounted read-only at $MOUNT_POINT"
echo ""
info "You can now copy files from the volume."
info "When done, unmount with: sudo diskutil unmount '$MOUNT_POINT'"
else
err "mount_exfat failed. Check that Full Disk Access is enabled for your terminal."
exit 1
fi
exit 0
fi
# ─── Full repair flow ────────────────────────────────────────────────────────
step 1 "Diagnose"
info "Force-unmounting $WHOLE_DISK..."
diskutil unmountDisk force "$WHOLE_DISK" 2>/dev/null || true
sleep 1
# Kill any lingering fsck_exfat processes
if pgrep -x fsck_exfat &>/dev/null; then
warn "fsck_exfat is running (spawned by diskarbitrationd). Waiting for it to finish..."
# Wait up to 5 minutes for fsck to complete
for i in $(seq 1 60); do
if ! pgrep -x fsck_exfat &>/dev/null; then
break
fi
sleep 5
done
if pgrep -x fsck_exfat &>/dev/null; then
warn "fsck_exfat still running after 5 minutes. Force-unmounting again..."
diskutil unmountDisk force "$WHOLE_DISK" 2>/dev/null || true
sleep 2
fi
fi
info "Reading boot sector from $RAW_DEVICE..."
# Use Python to parse the boot sector and diagnose
DIAG_OUTPUT=$(python3 << 'PYEOF'
import struct, sys, json
device = sys.argv[1] if len(sys.argv) > 1 else ""
if not device:
print("ERROR: No device specified")
sys.exit(1)
with open(device, 'rb') as f:
boot = f.read(512)
# Verify signature
sig = boot[3:11]
if sig != b'EXFAT ':
print(json.dumps({"error": f"Not an ExFAT filesystem (signature: {sig})"}))
sys.exit(1)
fields = {}
fields['fat_offset'] = struct.unpack_from('<I', boot, 0x50)[0]
fields['fat_length'] = struct.unpack_from('<I', boot, 0x54)[0]
fields['cluster_heap_offset'] = struct.unpack_from('<I', boot, 0x58)[0]
fields['cluster_count'] = struct.unpack_from('<I', boot, 0x5C)[0]
fields['root_dir_cluster'] = struct.unpack_from('<I', boot, 0x60)[0]
fields['volume_flags'] = struct.unpack_from('<H', boot, 0x6A)[0]
fields['bytes_per_sector_shift'] = boot[0x6C]
fields['sectors_per_cluster_shift'] = boot[0x6D]
fields['num_fats'] = boot[0x6E]
fields['percent_in_use'] = boot[0x70]
fields['bytes_per_sector'] = 1 << boot[0x6C]
fields['sectors_per_cluster'] = 1 << boot[0x6D]
fields['bytes_per_cluster'] = fields['bytes_per_sector'] * fields['sectors_per_cluster']
fields['dirty'] = bool(fields['volume_flags'] & 0x02)
fields['volume_size_gb'] = round((fields['cluster_count'] * fields['bytes_per_cluster']) / (1024**3), 1)
print(json.dumps(fields))
PYEOF
-- "$RAW_DEVICE" 2>&1) || {
err "Failed to read boot sector. Is Full Disk Access enabled for your terminal?"
err "System Preferences → Privacy & Security → Full Disk Access → add Terminal.app"
exit 1
}
# Check for parse error
if echo "$DIAG_OUTPUT" | python3 -c "import sys,json; d=json.load(sys.stdin); sys.exit(1 if 'error' in d else 0)" 2>/dev/null; then
ERROR_MSG=$(echo "$DIAG_OUTPUT" | python3 -c "import sys,json; print(json.load(sys.stdin)['error'])")
err "$ERROR_MSG"
exit 1
fi
# Extract fields
get_field() { echo "$DIAG_OUTPUT" | python3 -c "import sys,json; print(json.load(sys.stdin)['$1'])"; }
BYTES_PER_SECTOR=$(get_field bytes_per_sector)
SECTORS_PER_CLUSTER=$(get_field sectors_per_cluster)
BYTES_PER_CLUSTER=$(get_field bytes_per_cluster)
CLUSTER_COUNT=$(get_field cluster_count)
FAT_OFFSET=$(get_field fat_offset)
FAT_LENGTH=$(get_field fat_length)
CLUSTER_HEAP_OFFSET=$(get_field cluster_heap_offset)
ROOT_DIR_CLUSTER=$(get_field root_dir_cluster)
VOLUME_FLAGS=$(get_field volume_flags)
IS_DIRTY=$(get_field dirty)
PERCENT_IN_USE=$(get_field percent_in_use)
VOLUME_SIZE_GB=$(get_field volume_size_gb)
echo " Filesystem: ExFAT"
echo " Volume size: ${VOLUME_SIZE_GB} GB"
echo " Bytes per sector: $BYTES_PER_SECTOR"
echo " Sectors per cluster: $SECTORS_PER_CLUSTER"
echo " Bytes per cluster: $BYTES_PER_CLUSTER"
echo " Cluster count: $CLUSTER_COUNT"
echo " FAT offset: sector $FAT_OFFSET"
echo " FAT length: $FAT_LENGTH sectors"
echo " Cluster heap offset: sector $CLUSTER_HEAP_OFFSET"
echo " Root directory cluster: $ROOT_DIR_CLUSTER"
echo " Volume flags: 0x$(printf '%04X' "$VOLUME_FLAGS")"
echo " Dirty flag: $IS_DIRTY"
echo " Percent in use: ${PERCENT_IN_USE}%"
if [[ "$IS_DIRTY" == "True" ]]; then
warn "Volume dirty flag is SET — consistent with a crash during write"
else
info "Volume dirty flag is clear"
fi
if [[ "$DRY_RUN" == true ]]; then
echo ""
step 2 "Bitmap analysis (dry run)"
else
echo ""
step 2 "Rebuild allocation bitmap from FAT"
fi
# Create temp directory for intermediate files
TMPDIR_FIX=$(mktemp -d /tmp/exfat-fix.XXXXXX)
trap 'rm -rf "$TMPDIR_FIX"' EXIT
info "Rebuilding bitmap from FAT using Python..."
python3 << PYEOF -- "$RAW_DEVICE" "$TMPDIR_FIX" "$DRY_RUN"
import struct, sys, os, json
device = "$RAW_DEVICE"
tmpdir = "$TMPDIR_FIX"
dry_run = "$DRY_RUN" == "true"
with open(device, 'rb') as f:
# Parse boot sector
boot = f.read(512)
fat_offset = struct.unpack_from('<I', boot, 0x50)[0]
fat_length = struct.unpack_from('<I', boot, 0x54)[0]
cluster_heap_offset = struct.unpack_from('<I', boot, 0x58)[0]
cluster_count = struct.unpack_from('<I', boot, 0x5C)[0]
root_dir_cluster = struct.unpack_from('<I', boot, 0x60)[0]
volume_flags = struct.unpack_from('<H', boot, 0x6A)[0]
bps_shift = boot[0x6C]
spc_shift = boot[0x6D]
bytes_per_sector = 1 << bps_shift
sectors_per_cluster = 1 << spc_shift
bytes_per_cluster = bytes_per_sector * sectors_per_cluster
# Read the entire FAT
print(f" Reading FAT ({fat_length} sectors, {fat_length * bytes_per_sector / (1024*1024):.0f} MB)...")
f.seek(fat_offset * bytes_per_sector)
fat_data = f.read(fat_length * bytes_per_sector)
# Build new allocation bitmap from FAT
print(f" Scanning {cluster_count:,} clusters...")
bitmap_size_bytes = (cluster_count + 7) // 8
new_bitmap = bytearray(bitmap_size_bytes)
allocated = 0
for ci in range(2, cluster_count + 2):
off = ci * 4
if off + 4 > len(fat_data):
break
entry = struct.unpack_from('<I', fat_data, off)[0]
if entry != 0x00000000:
bi = ci - 2
new_bitmap[bi // 8] |= (1 << (bi % 8))
allocated += 1
used_pct = (allocated / cluster_count) * 100
used_gb = (allocated * bytes_per_cluster) / (1024**3)
print(f" Allocated clusters: {allocated:,} / {cluster_count:,} ({used_pct:.1f}%)")
print(f" Used space: {used_gb:.1f} GB")
# Find bitmap location from root directory
root_sector = cluster_heap_offset + (root_dir_cluster - 2) * sectors_per_cluster
f.seek(root_sector * bytes_per_sector)
root_data = f.read(sectors_per_cluster * bytes_per_sector)
bitmap_cluster = None
bitmap_data_length = None
for i in range(0, len(root_data), 32):
if root_data[i] == 0x81: # Allocation Bitmap entry
bitmap_cluster = struct.unpack_from('<I', root_data, i + 20)[0]
bitmap_data_length = struct.unpack_from('<Q', root_data, i + 24)[0]
print(f" Bitmap location: cluster {bitmap_cluster}, length {bitmap_data_length} bytes")
break
if bitmap_cluster is None:
print("ERROR: Could not find Allocation Bitmap directory entry")
sys.exit(1)
bitmap_sector = cluster_heap_offset + (bitmap_cluster - 2) * sectors_per_cluster
# Read existing bitmap for comparison
f.seek(bitmap_sector * bytes_per_sector)
bitmap_sectors_count = (bitmap_data_length + bytes_per_sector - 1) // bytes_per_sector
existing_bitmap = f.read(bitmap_sectors_count * bytes_per_sector)
diff_bytes = sum(1 for i in range(min(len(new_bitmap), len(existing_bitmap))) if new_bitmap[i] != existing_bitmap[i])
print(f" Bitmap differences: {diff_bytes:,} bytes differ out of {bitmap_size_bytes:,}")
if diff_bytes == 0:
print(" Bitmap is already correct! No repair needed.")
# Write status for the shell script
with open(os.path.join(tmpdir, 'status.json'), 'w') as sf:
json.dump({"bitmap_ok": True, "bitmap_sector": bitmap_sector, "used_pct": int(used_pct)}, sf)
sys.exit(0)
# Pad to sector boundary
pad = (bytes_per_sector - (len(new_bitmap) % bytes_per_sector)) % bytes_per_sector
padded = bytes(new_bitmap) + b'\x00' * pad
# Save new bitmap to temp file
bitmap_path = os.path.join(tmpdir, 'new_bitmap.bin')
with open(bitmap_path, 'wb') as bf:
bf.write(padded)
# Prepare updated boot sector with cleared dirty flag and updated percent-in-use
f.seek(0)
boot_data = bytearray(f.read(bytes_per_sector))
new_flags = volume_flags & ~0x02 # Clear dirty flag (bit 1)
struct.pack_into('<H', boot_data, 0x6A, new_flags)
boot_data[0x70] = int(used_pct)
# Calculate VBR checksum (sectors 0-10)
# We need to read sectors 1-10 as well
f.seek(bytes_per_sector)
rest_of_vbr = f.read(bytes_per_sector * 10)
vbr_data = bytes(boot_data) + rest_of_vbr
checksum = 0
for i in range(bytes_per_sector * 11):
# Skip VolumeFlags (106-107) and PercentInUse (112) per spec
if i == 106 or i == 107 or i == 112:
continue
checksum = ((checksum << 31) | (checksum >> 1)) + vbr_data[i]
checksum &= 0xFFFFFFFF
# Build checksum sector (sector 11): repeated uint32
checksum_sector = struct.pack('<I', checksum) * (bytes_per_sector // 4)
# Save boot sector and checksum sector
boot_path = os.path.join(tmpdir, 'new_boot.bin')
with open(boot_path, 'wb') as bf:
bf.write(bytes(boot_data))
cksum_path = os.path.join(tmpdir, 'new_checksum.bin')
with open(cksum_path, 'wb') as cf:
cf.write(checksum_sector)
# Also prepare backup boot region (starts at sector 12)
# Backup boot sector at sector 12, backup checksum at sector 23
backup_boot_path = os.path.join(tmpdir, 'backup_boot.bin')
with open(backup_boot_path, 'wb') as bf:
bf.write(bytes(boot_data))
backup_cksum_path = os.path.join(tmpdir, 'backup_checksum.bin')
with open(backup_cksum_path, 'wb') as cf:
cf.write(checksum_sector)
# Write status for shell script
with open(os.path.join(tmpdir, 'status.json'), 'w') as sf:
json.dump({
"bitmap_ok": False,
"bitmap_sector": bitmap_sector,
"bitmap_sectors": len(padded) // bytes_per_sector,
"bitmap_bytes": len(padded),
"used_pct": int(used_pct),
"diff_bytes": diff_bytes,
"new_flags": new_flags,
"checksum": checksum,
"bytes_per_sector": bytes_per_sector,
}, sf)
print(" Files prepared for writing.")
PYEOF
# Read status from Python
if [[ ! -f "$TMPDIR_FIX/status.json" ]]; then
err "Bitmap analysis failed"
exit 1
fi
BITMAP_OK=$(python3 -c "import json; print(json.load(open('$TMPDIR_FIX/status.json'))['bitmap_ok'])")
BITMAP_SECTOR=$(python3 -c "import json; print(json.load(open('$TMPDIR_FIX/status.json'))['bitmap_sector'])")
USED_PCT=$(python3 -c "import json; print(json.load(open('$TMPDIR_FIX/status.json'))['used_pct'])")
if [[ "$BITMAP_OK" == "True" ]]; then
ok "Allocation bitmap is already correct"
info "The bitmap matches the FAT — no repair needed."
info "The issue may be the dirty flag or another metadata problem."
fi
if [[ "$DRY_RUN" == true ]]; then
echo ""
info "Dry run complete. No changes were written."
if [[ "$BITMAP_OK" != "True" ]]; then
DIFF_BYTES=$(python3 -c "import json; print(json.load(open('$TMPDIR_FIX/status.json'))['diff_bytes'])")
BITMAP_BYTES=$(python3 -c "import json; print(json.load(open('$TMPDIR_FIX/status.json'))['bitmap_bytes'])")
echo ""
info "Summary of what --dry-run would fix:"
echo " • Bitmap: $DIFF_BYTES bytes differ (would write $BITMAP_BYTES bytes)"
echo " • Dirty flag: would be cleared"
echo " • VBR checksum: would be recalculated"
echo " • Percent-in-use: would be updated to ${USED_PCT}%"
fi
echo ""
info "To perform the repair, run without --dry-run:"
info " sudo bash $0 $DEVICE"
exit 0
fi
if [[ "$BITMAP_OK" != "True" ]]; then
step 3 "Write repaired bitmap"
BITMAP_SECTORS=$(python3 -c "import json; print(json.load(open('$TMPDIR_FIX/status.json'))['bitmap_sectors'])")
BITMAP_BYTES=$(python3 -c "import json; print(json.load(open('$TMPDIR_FIX/status.json'))['bitmap_bytes'])")
info "Writing rebuilt allocation bitmap ($BITMAP_BYTES bytes) to $RAW_DEVICE..."
info " Destination: sector $BITMAP_SECTOR ($BITMAP_SECTORS sectors)"
dd if="$TMPDIR_FIX/new_bitmap.bin" of="$RAW_DEVICE" \
bs=512 seek="$BITMAP_SECTOR" count="$BITMAP_SECTORS" conv=notrunc 2>&1 | while read -r line; do
echo " $line"
done
if [[ ${PIPESTATUS[0]} -ne 0 ]]; then
err "Failed to write bitmap. Check USB connection and Full Disk Access."
exit 1
fi
ok "Bitmap written successfully"
fi
step 4 "Clear dirty flag and update VBR checksum"
info "Writing updated boot sector (dirty flag cleared, percent-in-use updated)..."
dd if="$TMPDIR_FIX/new_boot.bin" of="$RAW_DEVICE" \
bs=512 seek=0 count=1 conv=notrunc 2>&1 | while read -r line; do
echo " $line"
done
ok "Boot sector updated"
info "Writing VBR checksum sector (sector 11)..."
dd if="$TMPDIR_FIX/new_checksum.bin" of="$RAW_DEVICE" \
bs=512 seek=11 count=1 conv=notrunc 2>&1 | while read -r line; do
echo " $line"
done
ok "VBR checksum updated"
info "Updating backup boot region (sector 12 + sector 23)..."
dd if="$TMPDIR_FIX/backup_boot.bin" of="$RAW_DEVICE" \
bs=512 seek=12 count=1 conv=notrunc 2>&1 | while read -r line; do
echo " $line"
done
dd if="$TMPDIR_FIX/backup_checksum.bin" of="$RAW_DEVICE" \
bs=512 seek=23 count=1 conv=notrunc 2>&1 | while read -r line; do
echo " $line"
done
ok "Backup boot region updated"
step 5 "Mount volume"
info "Attempting mount via diskutil..."
if diskutil mount "$DEVICE" 2>/dev/null; then
MOUNT_RESULT=$(mount | grep "$DEVICE" | awk '{print $3}')
ok "Volume mounted read-write at $MOUNT_RESULT"
else
warn "diskutil mount failed (fsck may still be unhappy). Falling back to mount_exfat..."
# Determine mount point
if [[ -z "$MOUNT_POINT" ]]; then
VOL_NAME=$(diskutil info "$DEVICE" 2>/dev/null | grep "Volume Name" | sed 's/.*: *//' || echo "ExFAT_Recovery")
VOL_NAME=$(echo "$VOL_NAME" | tr -cd '[:alnum:]._-')
[[ -z "$VOL_NAME" ]] && VOL_NAME="ExFAT_Recovery"
MOUNT_POINT="/Volumes/$VOL_NAME"
fi
diskutil unmountDisk force "$WHOLE_DISK" 2>/dev/null || true
sleep 1
mkdir -p "$MOUNT_POINT"
if mount_exfat "$DEVICE" "$MOUNT_POINT" 2>/dev/null; then
ok "Volume mounted read-write at $MOUNT_POINT (via mount_exfat)"
elif mount_exfat -o rdonly "$DEVICE" "$MOUNT_POINT" 2>/dev/null; then
warn "Mounted read-only at $MOUNT_POINT (read-write mount failed)"
warn "There may be additional filesystem damage beyond the bitmap."
else
err "All mount attempts failed."
err "Try manually: sudo mount_exfat -o rdonly $DEVICE /Volumes/Recovery"
exit 1
fi
fi
# Detect actual mount point
ACTUAL_MOUNT=$(mount | grep "$DEVICE" | awk '{print $3}' | head -1)
if [[ -z "$ACTUAL_MOUNT" ]]; then
ACTUAL_MOUNT="$MOUNT_POINT"
fi
step 6 "Disable Spotlight indexing"
info "Disabling Spotlight on $ACTUAL_MOUNT to prevent indexing storms..."
if mdutil -i off "$ACTUAL_MOUNT" 2>/dev/null; then
ok "Spotlight indexing disabled"
else
warn "Could not disable Spotlight (non-critical)"
fi
# ─── Summary ─────────────────────────────────────────────────────────────────
echo ""
echo -e "${GREEN}${BOLD}Recovery complete!${NC}"
echo ""
echo " Volume mounted at: $ACTUAL_MOUNT"
echo " Percent in use: ${USED_PCT}%"
echo ""
info "Verify your files, then consider running Disk Utility's First Aid"
info "to let macOS do a full consistency check now that the bitmap is fixed."
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment