+README.md

Restic Backup

Clone

clone to some location like /opt/restic:

git clone https://gist.github.com/Alistair1231/c3bfa81d2c3016a7138b2839d32996a8 /opt/restic

macOS

Linux is the primary target, but everything here also runs on macOS. Install the missing pieces via Homebrew first:

brew install bash flock coreutils jq restic

• bash — macOS ships bash 3.2; the scripts use associative arrays and mapfile, which need bash >= 4.
• flock — macOS has no built-in flock(1); restic.sh uses it for the run lock.
• coreutils — provides gdate (GNU date), which find-changed.sh uses for parsing relative/absolute times. macOS's built-in date is BSD date and doesn't support the same flags.
• jq, restic — same as Linux.

restic.sh picks up restic from PATH automatically, so no extra config is needed there. restic.exclude.example is written for a Linux/Steam setup (/home/*/.local/share/Steam/..., flatpak, etc.); those patterns just won't match anything on macOS, so adapt restic.exclude to your own paths instead (e.g. ~/Library/Caches, ~/Library/Application Support/Steam).

Configure

• Add a .restic-password file in the same directory as restic.sh with your repository password.
• Copy the restic.env.example and restic.exclude.example files and modify to your liking:

cp restic.env.example restic.env
vim restic.env

cp restic.exclude.example restic.exclude
vim restic.exclude

Init the repository

# source the environment variables
source restic.env

# init the repository
restic init

Automate Backups

Linux

Install the dependencies (e.g. on Arch Linux):

sudo pacman -S restic jq cronie

Add a crontab entry to run the backup script:

# enable cronie if not already running
sudo systemctl enable --now cronie

# then, either as current user
sudo EDITOR=vim crontab -u $USER -e
# or as root
sudo EDITOR=vim crontab -e

SHELL=/bin/bash
PATH=/bin:/usr/bin:/usr/local/bin

# Syntax:
# An example of setting a task through the scheduler:
# .---------------- minutes (0 - 59)
# |  .------------- hours (0 - 23)
# |  |  .---------- days (1 - 31)
# |  |  |  .------- months (1 - 12) OR may,jun,sept,aug ...
# |  |  |  |  .---- weekdays (0 - 6) OR mon,tue,wed,thu,fri,sat,sun
# |  |  |  |  |
# *  *  *  *  * the command you want to schedule
# test it out: https://crontab.guru/

# every 15 minutes, call the backup script and log the output. To view logs, run `journalctl -t restic`
*/15 * * * * /opt/restic/restic.sh 2>&1 | logger -t restic --prio-prefix

macOS

MacOS ships BSD cron, so the same crontab -e approach works:

# then, either as current user
sudo EDITOR=vim crontab -u $USER -e
# or as root
sudo EDITOR=vim crontab -e

# Homebrew's bin dir goes first so restic/jq/flock/gdate/bash are found.
# Apple Silicon: /opt/homebrew/bin — Intel: /usr/local/bin
PATH=/opt/homebrew/bin:/usr/local/bin:/bin:/usr/bin

# every 15 minutes, call the backup script and log the output.
# macOS's logger has no --prio-prefix, so it's left off. View logs with
# `log show --predicate 'eventMessage contains "restic"'` or search
# "restic" in Console.app.
*/15 * * * * /opt/restic/restic.sh 2>&1 | logger -t restic

cron needs Full Disk Access to read protected folders (Documents, Desktop, Downloads, Pictures, etc.), or backups of those will silently come back incomplete. Grant it once in System Settings → Privacy & Security → Full Disk Access → add /usr/sbin/cron (press Cmd+Shift+G in the file picker to type the path).

Restore

You can restore using restic mount, e.g.:

cd /opt/restic
source restic.env
restic mount /mnt/restic

afterwards, just use cp to copy the files you want to restore.

For other options and more info check the restore-docs

.gitignore

/.restic-password
/restic.env
/restic.exclude
/restic-repo
.last-maintenance
.last-success
.lock

find-changed.sh

#!/usr/bin/env bash
set -euo pipefail

# mapfile below needs bash >= 4. macOS ships bash 3.2; get bash from
# Homebrew instead (`brew install bash`).
if ((BASH_VERSINFO[0] < 4)); then
    echo "error: bash >= 4 required (macOS: brew install bash)" >&2
    exit 1
fi

# GNU date is needed for -d/-f below. Linux's system `date` already is GNU
# date; macOS's isn't, so prefer `gdate` from Homebrew coreutils there
# (`brew install coreutils`).
DATE_BIN=$(command -v gdate || command -v date)

usage() {
    cat <<'EOF'
Usage:
  restic-file-changes FILE [--from TIME] [--to TIME]

Show backups in which FILE changed according to its metadata.

Arguments:
  FILE          Path to the file inside the restic repository

Options:
  --from TIME   Only consider backups at or after TIME
  --to TIME     Only consider backups at or before TIME
  -h, --help    Show this help

TIME examples:
  2026-09-01
  2026-09-01 12:00:00
  7d
  24h
  2w
EOF
}

[[ $# -gt 0 ]] || {
    usage >&2
    exit 2
}

FILE=$1
shift

FROM=
TO=

while [[ $# -gt 0 ]]; do
    case "$1" in
        --from)
            [[ $# -ge 2 ]] || {
                echo "error: --from requires an argument" >&2
                exit 2
            }
            FROM=$2
            shift 2
            ;;
        --to)
            [[ $# -ge 2 ]] || {
                echo "error: --to requires an argument" >&2
                exit 2
            }
            TO=$2
            shift 2
            ;;
        -h|--help)
            usage
            exit 0
            ;;
        *)
            echo "error: unknown argument: $1" >&2
            exit 2
            ;;
    esac
done

# Convert an absolute date/time or a relative "Nd"/"Nh"/"Nw" shorthand to
# epoch seconds. GNU date doesn't accept "7d"/"24h"/"2w" on its own, so
# those are rewritten to "-N days"/"-N hours"/"-N weeks" first.
to_epoch() {
    local val=$1
    if [[ $val =~ ^[0-9]+d$ ]]; then
        val="-${val%d} days"
    elif [[ $val =~ ^[0-9]+h$ ]]; then
        val="-${val%h} hours"
    elif [[ $val =~ ^[0-9]+w$ ]]; then
        val="-${val%w} weeks"
    fi
    "$DATE_BIN" -d "$val" +%s
}

FROM_EPOCH=
TO_EPOCH=
if [[ -n "$FROM" ]]; then
    FROM_EPOCH=$(to_epoch "$FROM") || {
        echo "error: invalid --from time: $FROM" >&2
        exit 2
    }
fi
if [[ -n "$TO" ]]; then
    TO_EPOCH=$(to_epoch "$TO") || {
        echo "error: invalid --to time: $TO" >&2
        exit 2
    }
fi

# `restic find --json` reports each match's snapshot ID but not that
# snapshot's creation time, and has no --from/--to of its own (only
# -N/-O, which filter by the file's own mtime). So snapshot times are
# fetched separately, converted to epoch in one batch (date -f -, since
# jq's strptime/mktime silently ignores the UTC offset and gets this
# wrong), and only snapshots inside the requested window are kept in the
# id -> {short_id, time} map used below to join and filter at once.
snapshots_json=$(restic snapshots --json)

mapfile -t snap_ids < <(jq -r '.[].id' <<<"$snapshots_json")
mapfile -t snap_short_ids < <(jq -r '.[].short_id' <<<"$snapshots_json")
mapfile -t snap_iso_times < <(jq -r '.[].time' <<<"$snapshots_json")
mapfile -t snap_epochs < <(jq -r '.[].time' <<<"$snapshots_json" | "$DATE_BIN" -f - +%s)

snap_rows=""
for i in "${!snap_ids[@]}"; do
    epoch=${snap_epochs[$i]}
    [[ -n "$FROM_EPOCH" && "$epoch" -lt "$FROM_EPOCH" ]] && continue
    [[ -n "$TO_EPOCH" && "$epoch" -gt "$TO_EPOCH" ]] && continue
    snap_rows+="${snap_ids[$i]}"$'\t'"${snap_short_ids[$i]}"$'\t'"${snap_iso_times[$i]}"$'\n'
done

snap_times_json=$(printf '%s' "$snap_rows" | jq -R -s '
    split("\n")
    | map(select(length > 0) | split("\t"))
    | map({(.[0]): {short_id: .[1], time: .[2]}})
    | add // {}
')

restic find --json "$FILE" |
jq -c --arg file "$FILE" --argjson snaps "$snap_times_json" '
    .[]
    | .snapshot as $snap
    | ($snaps[$snap]) as $s
    | select($s != null)
    | .matches[]
    | select(.path == $file)
    | {
        time: $s.time,
        snapshot: $s.short_id,
        mtime: .mtime,
        size: .size
    }
' |
jq -rs '
    sort_by(.time)
    | . as $all
    | range(0; length)
    | . as $i
    | $all[$i] as $cur
    | if $i == 0 then
          $cur
      elif (
          $cur.mtime != $all[$i - 1].mtime
          or $cur.size != $all[$i - 1].size
      ) then
          $cur
      else
          empty
      end
    | [
        .time,
        .snapshot,
        .mtime,
        .size
      ]
    | @tsv
'

restic.env.example

# --- Backend: pick ONE block below and fill it in --------------------------

# -- local --
# export RESTIC_REPOSITORY="/path/to/repo"

# -- sftp --
export RESTIC_REPOSITORY="sftp:user@host:/path"
# Only needed if not relying on ~/.ssh/config or an agent:
export SSH_KEY="/path/to/key"

# -- rest-server --
# export RESTIC_REPOSITORY="rest:http://user:pass@host:8000/repo"

# -- S3 / S3-compatible (MinIO, etc.) --
# export RESTIC_REPOSITORY="s3:host/bucket"
# export AWS_ACCESS_KEY_ID="..."
# export AWS_SECRET_ACCESS_KEY="..."

# -- Backblaze B2 --
# export RESTIC_REPOSITORY="b2:bucket:path"
# export B2_ACCOUNT_ID="..."
# export B2_ACCOUNT_KEY="..."

# -- rclone (any rclone remote: Google Drive, Dropbox, OneDrive, etc.) --
# export RESTIC_REPOSITORY="rclone:remote:path"
# export RCLONE_CONFIG="/etc/rclone/rclone.conf"

# --- Paths to back up, one per line, "quoted" --------------------------------------
export BACKUP_TARGETS=(
    "$HOME"
    "/etc"
)

# --- Retention policy: each is independent, comment out any you don't ------
# --- want enforced. If all four are commented out, forget/prune is --------
# --- skipped entirely (check still runs). ----------------------------------
export KEEP_LAST=24
export KEEP_DAILY=7
export KEEP_WEEKLY=4
export KEEP_MONTHLY=2

# --- Optional overrides -----------------------------------------------------
# All of these are optional; restic.sh falls back to sensible defaults if
# left commented out.

# Password file to use for the backup. Defaults to .restic-password next to
# restic.sh
# export RESTIC_PASSWORD_FILE="/opt/restic/.restic-password"

# Exclude patterns for the backup. Defaults to restic.exclude next to
# restic.sh (i.e. in the same directory as this file).
# export RESTIC_EXCLUDE_FILE=/opt/restic/restic.exclude

# The restic binary to run. Defaults to /usr/bin/restic.
# export RESTIC_BIN=/usr/bin/restic

# Hour (24h clock, 00-23) at or after which the daily maintenance step
# (forget/prune + check) is allowed to run. Maintenance runs at most once
# per calendar day, on the first run at or after this hour — so a machine
# that's off at exactly this hour still catches up later the same day.
# Defaults to 03.
# export MAINTENANCE_HOUR=03

# File used to remember the date maintenance last ran, so it isn't repeated
# every hour. Defaults to .last-maintenance next to restic.sh.
# export MAINTENANCE_STATE_FILE=/opt/restic/.last-maintenance

restic.exclude.example

# various big stuff
/home/*/.local/share/Steam/steamapps
/home/*/.local/share/Steam/userdata/*/gamerecordings
/home/*/.local/share/Steam/compatibilitytools.d
/home/*/.local/share/flatpak
/home/*/.local/share/Trash
/home/*/.local/share/bottles
/home/*/.local/share/containers
/home/*/winboat
/home/*/Mods
/home/*/Videos
/home/*/Music/Libation
/home/*/Downloads
/home/*/.ftba
/home/*/.vscode

# programming stuff
/home/*/.cargo/registry
/home/*/.cargo/git
/home/*/.gradle/caches
/home/*/.m2/repository
/home/*/.npm/_cacache
/home/*/.ccache

# Cache and log files
/home/*/**/*Cache*
/home/*/**/*cache*
/home/*/.cache
/home/*/.cache/**
/home/*/.local/state
/home/*/.local/state/**
/home/*/.log
/home/*/.log/**

# Flatpak
/home/*/.var/app/moe.launcher.sleepy-launcher/data/sleepy-launcher/
/home/*/.var/app/*/cache
/home/*/.var/app/*/.cache
/home/*/.var/app/*/tmp
/home/*/.var/app/*/.tmp
/home/*/.var/app/*/log
/home/*/.var/app/*/.local/state

restic.sh

#!/usr/bin/env bash
set -euo pipefail
trap '<3>echo "!!!!!!!! BACKUP FAILED (line $LINENO, exit $?) $(date) !!!!!!!!"' ERR

# Associative arrays below need bash >= 4. macOS ships bash 3.2; get bash
# from Homebrew instead (`brew install bash`).
if ((BASH_VERSINFO[0] < 4)); then
    echo "error: bash >= 4 required (macOS: brew install bash)" >&2
    exit 1
fi

# Directory this script lives in, so companion files are found next to it
# rather than hardcoded. Each can still be overridden via env var.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

# get exclusive access to the lock file while running
exec 9>"${LOCK_FILE:-$SCRIPT_DIR/.lock}"
if ! flock -n 9; then
    echo "previous run still active, skipping $(date)"
    exit 0
fi

source "${RESTIC_ENV_FILE:-$SCRIPT_DIR/restic.env}"

RESTIC_BIN="${RESTIC_BIN:-$(command -v restic || echo /usr/bin/restic)}"
RESTIC_EXCLUDE_FILE="${RESTIC_EXCLUDE_FILE:-$SCRIPT_DIR/restic.exclude}"
if [ "${BACKUP_TARGETS+set}" != "set" ]; then
    BACKUP_TARGETS=("$HOME" /etc)
fi
MAINTENANCE_HOUR="${MAINTENANCE_HOUR:-03}"
MAINTENANCE_STATE_FILE="${MAINTENANCE_STATE_FILE:-$SCRIPT_DIR/.last-maintenance}"

log() { echo "========= $* $(date) ========="; }

# Backend-specific extra options. Only sftp needs one (an explicit key);
# other backends read credentials from the environment.
RESTIC_OPTS=()
if [ -n "${SSH_KEY:-}" ]; then
    RESTIC_OPTS+=(-o "sftp.args=-i $SSH_KEY")
fi

# Each retention flag is optional; unset means "don't enforce that cutoff".
# No default — if none are set, forget/prune is skipped below.
KEEP_OPTS=()
[ -n "${KEEP_LAST:-}" ] && KEEP_OPTS+=(--keep-last "$KEEP_LAST")
[ -n "${KEEP_DAILY:-}" ] && KEEP_OPTS+=(--keep-daily "$KEEP_DAILY")
[ -n "${KEEP_WEEKLY:-}" ] && KEEP_OPTS+=(--keep-weekly "$KEEP_WEEKLY")
[ -n "${KEEP_MONTHLY:-}" ] && KEEP_OPTS+=(--keep-monthly "$KEEP_MONTHLY")

log START

# Clear stale locks from a crashed/killed run. Safe: without --remove-all
# this only removes locks whose owner is dead. Non-fatal under `set -e`.
"$RESTIC_BIN" unlock "${RESTIC_OPTS[@]}" || true

# --retry-lock waits out brief overlap with a live run.
# --skip-if-unchanged needs a *relative* path: an absolute target embeds
# ancestor-directory mtimes too, and their churn defeats the skip. So we cd
# into each target's parent and pass the leaf name instead. Targets sharing
# a parent are grouped into one backup call (one snapshot); others get
# their own call.
declare -A targets_by_parent=()
for target in "${BACKUP_TARGETS[@]}"; do
    parent=$(dirname -- "$target")
    leaf=$(basename -- "$target")
    targets_by_parent["$parent"]+="$leaf"$'\n'
done

for parent in "${!targets_by_parent[@]}"; do
    leaves=()
    while IFS= read -r leaf; do
        [ -n "$leaf" ] && leaves+=("$leaf")
    done <<< "${targets_by_parent[$parent]}"
    (
        cd "$parent"
        rc=0
        "$RESTIC_BIN" -v backup "${leaves[@]}" \
            "${RESTIC_OPTS[@]}" \
            --retry-lock "5m" \
            --skip-if-unchanged \
            --exclude-file="$RESTIC_EXCLUDE_FILE" || rc=$?
        if [ "$rc" -eq 3 ]; then
            echo "<4>WARNING: backup of $parent incomplete, some files unreadable $(date)"
            exit 3
        elif [ "$rc" -ne 0 ]; then
            exit "$rc"
        fi
    )
done

# Daily (at or after MAINTENANCE_HOUR): forget, prune, check. I/O-heavy, so
# gated to once a day via >= and a last-run date file rather than an exact
# hour match, so a machine asleep at MAINTENANCE_HOUR still catches up later
# the same day. 10# prefixes force base-10 (08/09 aren't valid octal).
today="$(date +%F)"
last_maintenance="$(cat "$MAINTENANCE_STATE_FILE" 2>/dev/null || true)"
if [ "$((10#$(date +%H)))" -ge "$((10#$MAINTENANCE_HOUR))" ] && [ "$last_maintenance" != "$today" ]; then
    if [ "${#KEEP_OPTS[@]}" -gt 0 ]; then
        "$RESTIC_BIN" forget \
            "${RESTIC_OPTS[@]}" \
            --retry-lock "5m" \
            "${KEEP_OPTS[@]}" \
            --prune
    else
        log "no KEEP_* retention policy configured, skipping forget/prune"
    fi

    if [ "$(date +%u)" -eq 7 ]; then
        "$RESTIC_BIN" check "${RESTIC_OPTS[@]}" --retry-lock "5m" --read-data-subset=5%
    else
        "$RESTIC_BIN" check "${RESTIC_OPTS[@]}" --retry-lock "5m"
    fi

    # Only recorded after forget/check succeed, so a failure (set -e exits
    # first) makes the next run retry maintenance instead of skipping it.
    echo "$today" > "$MAINTENANCE_STATE_FILE"
fi

date '+%F %T' > "$SCRIPT_DIR/.last-success"
log END