Last active
March 1, 2025 16:56
-
-
Save coderjo/c8de3ecc31f1d6450254b5e97ea2c595 to your computer and use it in GitHub Desktop.
access a disk image read-only with a copy-on-write overlay to allow fsck or the like to write changes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# usage: attach_cow_image.sh [imagefile] [cowfile] [devname] | |
# imagefile: path to the image file you want to load | |
# cowfile: path to the file to store writes into. If it doesn't exist, a sparse 1GB file will be created. | |
# devname: the name you want the drive to show up as in /dev/mapper | |
imgfile="$1" | |
cowfile="$2" | |
dmname="$3" | |
# create COW file if it doesn't exist (otherwise, assume we are using a file from a previous use) | |
# the if you don't think 1000MiB will be enough, you can create a larger one. | |
[ -f "$cowfile" ] || dd if=/dev/zero of="$cowfile" bs=1 count=1 seek=1048576000 | |
# attach the files to loop devices (with the image file read-only) | |
imgdev=`losetup -f -r --show "$imgfile"` | |
cowdev=`losetup -f --show "$cowfile"` | |
# get the size of the image device | |
imgsz=`blockdev --getsz $imgdev` | |
# create the devmapper table for a copy-on-write device using the two loop devices | |
# p means persist the snapshot data | |
# The 4 is the number of sectors to use per COW image chunk | |
echo 0 $imgsz snapshot $imgdev $cowdev p 4| dmsetup create $dmname | |
# and now probe the partition table of the new device | |
partprobe -s /dev/mapper/$dmname | |
# to detatch everything: | |
# dmsetup remove $dmname | |
# losetup -d $cowdev | |
# losetup -d $imgdev |
I had disastrous performances on a 4TB hdd hosting origin img and cow files: ~3MiB/s:
# sync ; time sh -c "dd if=/dev/zero of=tmp-$(date +%s) bs=1M count=100 ; sync"
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0,114658 s, 915 MB/s
real 0m28,193s
user 0m0,003s
sys 0m0,119s
A quick research on chunk size showed:
- https://www.nikhef.nl/~dennisvd/lvmcrap.html talks about
64k
in ~2010, but I don't know if it's 64KiB or 64k sectors. - https://docs.redhat.com/fr/documentation/red_hat_enterprise_linux/9/html/configuring_and_managing_logical_volumes/overview-of-chunk-size_creating-and-managing-thin-provisioned-volumes#overview-of-chunk-size_creating-and-managing-thin-provisioned-volumes talks about lvm2 using 64KiB chunk size or bigger.
I tried with 128KiB chunk size (256 sectors) and it mostly fixed my performance issue:
# sync ; time sh -c "dd if=/dev/zero of=tmp-$(date +%s) bs=1M count=1000 ; sync"
1000+0 records in
1000+0 records out
1048576000 bytes (1,0 GB, 1000 MiB) copied, 1,28779 s, 814 MB/s
real 0m9,339s
user 0m0,003s
sys 0m1,268s
=> ~107MiB/s
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
You can use ddrescue with partclone domain map to only read used data.
Never run any recovery on failed disk, always work on bit-perfect copy.
If you don't want to recover any deleted files and just failing medium is your concern - you can use partclone
Create ddrescue domain log from source device
option to output the map containing only used space and then use ddrescue with map to copy the disk's used space. Then use this recovered copy with snapshot and play with it. Snapshot allows you to not lose source file which may be problematic to copy again due to medium failing.