Helper for punching holes into a disk image where partclone says that blocks are not in use

domain-map-2-punch-holes.py

#!/usr/bin/env python

import sys

offset = int(sys.argv[1])
img_name = sys.argv[2]

# skip current_pos
while True:
    line = sys.stdin.readline()
    if not line: break
    if line.startswith('#'): continue
    # reached first non-comment line
    # --> this is the current_pos line
    break

# process ranges
while True:
    line = sys.stdin.readline()
    if not line: break
    if line.startswith('#'): continue
    fields = line.split()
    if fields[2] == '+': continue
    assert fields[2] == '?'
    # found free area
    start = int(fields[0], 16)
    length = int(fields[1], 16)
    sys.stdout.write('fallocate -p -l %d -o %d %s\n' %(
        length, start + offset, img_name
    ))

Creates fallocate --punch-hole commands for each byte range reported as unused (status ?) by a domain mapfile (provided on stdin) such as produced by partclone.[fstype] -D. A non-zero offset can be specified to account for headers, e.g. when punching holes into a LUKS-encrypted container. If using with LVM or software RAID carefully check that the payload is indeed stored sequentially.

This can be used to make file areas sparse that are not in use by the filesystem in the image file, freeing disk space. For plain images, a frequently suggested alternative is to first write zero bytes to the free space (by writing large, non-sparse files), then using fallocate --dig-holes or cp --sparse=always to create a sparse image and finally deleting the zero files inside. For LUKS containers, this approach cannot work as one does not know what data one must write in order for a block of zero bytes to come out of the encryption.

Example:

losetup --find --show filesystem.img 
partclone.btrfs -D -s /dev/loop0 -o filesystem.map
../domain-map-2-punch-holes.py 0 filesystem.img < filesystem.map > punch-holes.sh
losetup -d /dev/loop0
bash ./punch-holes.sh 
losetup --find --show filesystem.img 
btrfs check /dev/loop0
mount /dev/loop0 /mnt
btrfs scrub /mnt

Known issue: Thomas-Tsai/partclone#180