`rsync_ext`: Sync Selected File Extensions Over SSH #SSH

doc.md

Usage

Sync a single extension

rsync_ext user@remote /remote/base/dir/ /local/target/dir/ cif

Sync multiple extensions

rsync_ext user@remote /remote/base/dir/ /local/target/dir/ cif json POSCAR

Extensions may be passed with or without leading dots:

rsync_ext user@remote /remote/base/dir/ /local/target/dir/ .cif .vasp .json

---

Using a Jump Host (ProxyJump)

rsync_ext user@compute /remote/base/dir/ /local/target/dir/ --jump user@login cif json

This is equivalent to using ssh -J user@login.

---

Behavior and Guarantees

• Only files matching the specified extensions are transferred
• Full directory hierarchy is preserved
• Empty directories are removed locally (--prune-empty-dirs)
• Transfers are resumable and incremental (standard rsync behavior)

---

Optional Tweaks

Add progress output:

rsync -av --info=progress2 ...

Dry run (recommended for first use):

rsync -avn ...

---

Typical Use Cases

• Collecting .cif, .vasp, or .json files from large HPC directory trees
• Selective data synchronization from compute nodes
• Reproducible, scriptable file harvesting over SSH

rsync_ext.sh

rsync_ext() {
    local remote="$1"        # user@host
    local remote_dir="$2"    # /remote/base/dir/
    local local_dir="$3"     # /local/target/dir/
    shift 3

    # Optional: if the next arg is --jump <jumpHost>
    local jump=""
    if [[ "${1:-}" == "--jump" ]]; then
        jump="${2:-}"
        shift 2
    fi

    if [[ $# -lt 1 ]]; then
        echo "Usage: rsync_ext user@host /remote/dir/ /local/dir/ [--jump user@login] ext1 [ext2 ...]"
        return 2
    fi

    local ssh_cmd="ssh"
    [[ -n "$jump" ]] && ssh_cmd="ssh -J $jump"

    # Build include rules: directories + each extension pattern
    local -a inc_args
    inc_args+=(--include='*/')
    local ext
    for ext in "$@"; do
        ext="${ext#.}"                 # strip leading dot if provided
        inc_args+=(--include="*.${ext}")
    done
    inc_args+=(--exclude='*')

    rsync -av --prune-empty-dirs \
        "${inc_args[@]}" \
        -e "$ssh_cmd" \
        "${remote}:${remote_dir}" "${local_dir}"
}