lvm_cache.md

Test disk speed

• https://cloud.google.com/compute/docs/disks/benchmarking-pd-performance

• write

# sequential write
sudo fio --name=write_throughput --directory=. --numjobs=4 \
      --size=2G --time_based --runtime=5m --ramp_time=2s --ioengine=libaio \
      --direct=1 --verify=0 --bs=1M --iodepth=64 --rw=write \
      --group_reporting=1 --iodepth_batch_submit=64 \
      --iodepth_batch_complete_max=64
      
# random write
sudo fio --name=write_iops --directory=. --size=2G \
      --time_based --runtime=5m --ramp_time=2s --ioengine=libaio --direct=1 \
      --verify=0 --bs=4K --iodepth=256 --rw=randwrite --group_reporting=1  \
      --iodepth_batch_submit=256  --iodepth_batch_complete_max=256

• read

# sequential read
sudo fio --name=read_throughput --directory=. --numjobs=4 \
      --size=2G --time_based --runtime=5m --ramp_time=2s --ioengine=libaio \
      --direct=1 --verify=0 --bs=1M --iodepth=64 --rw=read \
      --group_reporting=1 \
      --iodepth_batch_submit=64 --iodepth_batch_complete_max=64
      
# random read
sudo fio --name=read_iops --directory=. --size=2G \
      --time_based --runtime=5m --ramp_time=2s --ioengine=libaio --direct=1 \
      --verify=0 --bs=4K --iodepth=256 --rw=randread --group_reporting=1 \
      --iodepth_batch_submit=256  --iodepth_batch_complete_max=256

• write

dd if=/dev/zero of=/tmp/output bs=64k count=100k conv=fdatasync

• read

dd if=/tmp/output of=/dev/null bs=64k

Check storage blocks

lsblk

LVM

• https://www.digitalocean.com/community/tutorials/an-introduction-to-lvm-concepts-terminology-and-operations
• https://www.digitalocean.com/community/tutorials/how-to-use-lvm-to-manage-storage-devices-on-ubuntu-18-04#step-2-creating-or-extending-lvm-components

# scan block devices
sudo lvmdiskscan

# create physical volumn
sudo pvcreate /dev/sda /dev/sdb

# list physical volumn
sudo pvs

# create volumn group
sudo vgcreate lvms /dev/sda /dev/sdb

# list volumn group
sudo vgs

# create raid0-like striped volumn
sudo lvcreate --type striped -i 2 -L 10G -n lvmcache lvms

# format
sudo mkfs.ext4 /dev/lvms/lvmcache

# mount
sudo mkdir -p /mnt/lvmcache
sudo mount /dev/lvms/lvmcache /mnt/lvmcache

LVM cache

• https://man7.org/linux/man-pages/man7/lvmcache.7.html
• https://forum.level1techs.com/t/lvmcache-a-simple-how-to/133247

# create cache meta
sudo lvcreate -n meta -L 122m lvms /dev/nvme1n1
# create cache
sudo lvcreate -n cache -L 100g lvms /dev/nvme1n1
# create slow LV
sudo lvcreate -n ebs -l 100%FREE lvms /dev/nvme2n1

# create cache
sudo lvconvert --type cache-pool --poolmetadata lvms/meta lvms/cache
# map the cache
sudo lvconvert --type cache --cachepool lvms/cache lvms/ebs

# writethrough -> writeback
sudo lvchange --cachemode writeback lvms/ebs

GCP c3-standard-4-lssd

• using the local SSD

sequential write: 406MB/s
sequential read:  737MB/s
random write: 410MB/s
random read:  737MB/s

• default balanced persistent disk (block size is 512 cat /sys/block/<disk>/queue/physical_block_size)

sequential write: 176MB/s
sequential read:  176MB/s
random write: 14.7MB/s
random read:  14.7MB/s

• write to the persistent disk with local SSD cache (writeback)

sequential write: 411MB/s
sequential read:  634MB/s
random write: 32.5MB/s
random read:  539MB/s