Rsync Command Cheat Sheet

common-media-sync.md

Quick Adds

• Direction: Pull
• User: inanimate
• Remote module name <mod>/<path>/*
• Enabled
• Recursive
• Times
• Archive
• Preserve Extended Attributes
• Delay Updates
• -Pvihu --timeout=300 --chmod=ug=rwX,o=rX --ignore-missing-args
• (test first!) --dry-run
• (if needed) --ignore-existing
• (if needed) --log-file=/tmp/rsync.log

rsync \
  --archive \
  --xattrs \
  --update \
  --itemize-changes \
  --delay-updates \
  --partial \
  --progress \
  --human-readable \
  --verbose \
  --timeout=300 \
  --chmod=ug=rwX,o=rX \
  --dry-run

Optionally, add --ignore-existing if copying from NTFS to a "normal" filesystem.

I've added --ignore-missing-args as a needed flag to ensure when using a wildcard derp/* for the source, but which nothing exists inside of derp, rsync doesn't through an error.

rsync-local-and-network-reference.md

rsync --dry-run -auhviPHX /mnt/bar /mnt/foo/ explainshell

*This will result in bar existing in foo so you'd have /mnt/foo/bar/bar1.jpg

rsync --dry-run -auhviPHX /mnt/bar/* /mnt/foo/ explainshell

*This will result in solely the non-hidden contents of bar (bar/*) existing inside foo so you'd have /mnt/foo/bar1.jpg

Network Transfer Considerations (tailscale/sshfs/etc)

When using this to transfer over a network, consider using the following:

• --delay-updates which will ensure your active usecases aren't seeing temp/stub files
• --timeout=300 has rsync quit out if the connection is failing
• When referencing a module + path, to copy the contents of the directory, verify you use <module-name>/<dir>/
• To view modules or their contents, use rsync rsync://<hostname>/ or rsync <hostname>::, then build the path to view its contents.

Don't use compress! You don't need it and you'll incur extra cpu overhead which will impact performance

Setting Permissions for Destination

Use the --chmod=ug=rwX,o=rX flag to rsync to ensure 664 for files and 775 for directories (if you want group and user to have same permissions).

This executes post --perms (inherently included in --archive). If using --no-perms, it won't even consider the source perms, defaulting to the standard perms for the user but still executes the --chmod post transfer either way so you get the same result.

Handling Destination

Generally, you'll more than likely want --update for most cases unless you want a perfect 1:1 mirror (i.e. source is authoritative)

• --ignore-existing - Skip all existing files (never updates or overwrites them).
• --update (-u) - Skip files that exist and have a newer timestamp on the destination (file size not considered, still updates older).
• --existing - Only update files that already exist (ignores new files on the source).

Note: This might not work as expected for filesystems that handle timestamps differently, cough..NTFS

Man Pages

https://linux.die.net/man/1/rsync https://www.man7.org/linux/man-pages/man1/rsync.1.html

rsync-mirrors-reference.md

rsync --dry-run -ahviyHP --delete-delay --delay-updates --timeout=300 rsync://mirror.rit.edu/ubuntu-releases /mnt/isos/linux/ubuntu-releases explainshell

This will result in /mnt/isos/linux/ubuntu-releases receiving the contents (i.e. 24.10/, xenial/, etc.) of the module!

-a is the same as -rlptgoD

-P is --partial and --progress - just re-execute command if it times out!

Slashes and Wildcards Matter when defining the Source!

• /source/* (Preferred!) → only the non-hidden contents of the folder are copied
• /source/{*,.[!.]*,..?*} → all contents, including hidden, of the folder are copied
• /source/ → contents of the folder are copied, including the timestamp and permissions of the encasing folder source to the destination containing folder destination/
• /source → folder itself is copied into destination

This is different with referencing a rsync module! It will always sync the contents of the module, not the module name as a directory.

Example

rsync -a /mnt/usb1/data/ /mnt/pool/backup/     # Copies contents
rsync -a /mnt/usb1/data  /mnt/pool/backup/     # Creates /backup/data/

Bandwidth Limit in kbps

--bwlimit=25600 # ~ 25MBps

Any matches glob

--exclude='*live*'

Specific path

--exclude='somefolder/README.txt'

Specific top Directory

--exclude='temp/'

Exclude everything under a path (incl subdirs)

--exclude='temp/**'

Exclude all "thumbnails" directories matching anywhere

--exclude='*/thumbnails/'

Exclude by file extension

--exclude='*.mp4'