Safe abraunegg OneDrive resync for Linux — local-wins conflict resolution with btrfs snapshots

onedrive-safe-resync.py

#!/usr/bin/env python3
"""
Safe OneDrive resync — local wins, smart conflict resolution.

════════════════════════════════════════════════════════════════════════
PREREQUISITES (run once per machine)
════════════════════════════════════════════════════════════════════════

1. abraunegg onedrive installed and configured
     sudo dnf install onedrive
     onedrive --synchronize          # first-time auth + initial sync
     systemctl --user enable onedrive

2. ~/OneDrive must be a btrfs subvolume (not just a plain directory).
   Check:
     btrfs subvolume show ~/OneDrive 2>/dev/null || echo "needs conversion"

   Convert if needed:
     systemctl --user stop onedrive
     mv ~/OneDrive ~/OneDrive-old
     btrfs subvolume create ~/OneDrive
     rsync -av ~/OneDrive-old/ ~/OneDrive/
     btrfs subvolume show ~/OneDrive   # verify
     rm -rf ~/OneDrive-old
     systemctl --user start onedrive

   Why: btrfs snapshots only work on subvolumes, not plain directories.
   Snapshots are instant and zero-copy at creation (CoW) — they only
   consume space for files that change after the snapshot is taken.

3. Passwordless sudo for btrfs subvolume commands:
     sudo EDITOR=joe visudo -f /etc/sudoers.d/<username>-btrfs

   Add these two lines (Ctrl+K X to save in joe):
     <username> ALL=(root) NOPASSWD: /usr/sbin/btrfs subvolume snapshot *
     <username> ALL=(root) NOPASSWD: /usr/sbin/btrfs subvolume delete *

════════════════════════════════════════════════════════════════════════
HOW IT WORKS
════════════════════════════════════════════════════════════════════════

Problem: abraunegg --resync treats remote as absolute truth. It deletes
local files not on remote and overwrites local edits with remote versions.
This destroys work done locally during a sync outage.

Solution:
  1. Snapshot ~/OneDrive before resync (instant, captures local state)
  2. Run resync (remote wins — downloads phone photos, other device files)
  3. Compare snapshot (your local truth) vs post-resync (remote truth)
  4. Restore your local versions; save displaced remote versions to
     ~/OneDrive-conflicts/ for manual review

Key: abraunegg's sqlite DB mtime = timestamp of last successful sync.
Files modified after that timestamp were created/edited during the outage
and are treated as your local work. Files older than that were already
synced — if remote deleted them, we respect that decision.

Decision logic (with working sqlite):
  Both exist, differ         → local wins, remote version → conflicts/
  Local only, mtime > sync   → restore (your work during outage)
  Local only, mtime < sync   → skip (remote deleted it, respect that)
  Remote only, mtime > sync  → keep (new file from another device/phone)
  Remote only, mtime ≤ sync  → remove, save remote version → conflicts/

Fallback when sqlite is missing or corrupt (last_sync = 0):
  REMOTE_WINS dirs           → keep (protects Android photo folders)
  Everything else            → local wins, remote → conflicts/

════════════════════════════════════════════════════════════════════════
DISK SPACE NOTES
════════════════════════════════════════════════════════════════════════

Snapshots (~/.onedrive-snapshots/):
  CoW — initially zero space, grows only for files changed after snapshot.
  Auto-pruned to MAX_SNAPSHOTS. Safe to increase for longer history.
  Check usage: sudo btrfs qgroup show -reF /home  (enable first if needed:
               sudo btrfs quota enable /home)

Conflict dirs (~/OneDrive-conflicts/):
  Full file copies — NOT CoW. Each resync session creates one directory
  containing full copies of remote files that lost to local versions.
  Size = sum of overwritten/removed remote files from that session.
  Auto-pruned to MAX_CONFLICTS. Delete manually when done reviewing.

Recovery:
  Wrongly removed remote file:  cp ~/OneDrive-conflicts/<ts>/path ~/OneDrive/path
  Wrongly skipped local file:   cp ~/.onedrive-snapshots/<ts>/path ~/OneDrive/path
  Web recycle bin (30 days):    https://onedrive.live.com/?id=recycle
"""

import filecmp
import shutil
import subprocess
import sys
from datetime import datetime
from pathlib import Path


# ── Config ──────────────────────────────────────────────────

HOME     = Path.home()

# abraunegg sync directory
SYNC_DIR = HOME / "OneDrive"

# where btrfs snapshots are stored (hidden dir, same btrfs filesystem as HOME)
SNAP_DIR = HOME / ".onedrive-snapshots"

# where remote versions of overwritten/removed files are saved for review
# one timestamped subdir per resync session; contains full file copies (not CoW)
CONFLICT_DIR = HOME / "OneDrive-conflicts"

# abraunegg sqlite database — its mtime = timestamp of last successful sync
# used to distinguish "your new work" (mtime > last sync) from
# "old synced file that remote deleted" (mtime < last sync)
DB_PATH = HOME / ".config/onedrive/items.sqlite3"

# Android photo folders under ~/OneDrive/Pictures/ — only used as fallback
# when sqlite is missing/corrupt and we can't use mtime to decide.
# With working sqlite, mtime logic handles these correctly without this list.
# Update when new Android apps create new photo folders.
REMOTE_WINS = {
    # Add subdirectories under ~/OneDrive/Pictures/ that your Android
    # device uploads to. Only used as fallback when sqlite is missing.
    "Pictures/DCIM",
    "Pictures/Pictures",
    "Pictures/Screenshots",
    "Pictures/Uploads",
    # "Pictures/Telegram",
    # "Pictures/WhatsApp Images",
    # add more as needed
}

# how many btrfs snapshots to keep before pruning oldest
# snapshots are CoW — low cost, can keep more if space allows
MAX_SNAPSHOTS = 5

# how many conflict dirs to keep before pruning oldest
# conflict dirs are full file copies — each can be MBs to GBs
# delete manually after reviewing if you want to free space sooner
MAX_CONFLICTS = 10


# ── Logger ──────────────────────────────────────────────────

class ResyncLog:
    ACTIONS = [
        "restored", "overwritten", "removed",
        "kept_remote", "skipped", "unchanged",
    ]

    def __init__(self, path, last_sync_str):
        path.parent.mkdir(parents=True, exist_ok=True)
        self.path = path
        self.file = open(path, "w")
        self.counts = {a: 0 for a in self.ACTIONS}

        header = (
            f"OneDrive Safe Resync — {datetime.now():%d.%m.%Y %H:%M:%S}\n"
            f"Last sync: {last_sync_str}\n"
            f"{'═' * 70}\n\n"
        )
        self.file.write(header)
        print(header, end="")

    def log(self, action, rel, detail=""):
        self.counts[action] = self.counts.get(action, 0) + 1

        # "unchanged" would spam thousands of lines — counted but not listed
        if action == "unchanged":
            return

        line = f"[{action:14s}] {rel}"
        if detail:
            line += f"  ({detail})"

        print(f"  {line}")
        self.file.write(f"{line}\n")

    def write_summary(self):
        sep = f"\n{'═' * 70}\nSummary:\n"
        print(sep, end="")
        self.file.write(sep)

        for action in self.ACTIONS:
            count = self.counts[action]
            if count > 0:
                line = f"  {action:14s}  {count}\n"
                print(line, end="")
                self.file.write(line)

    def close(self):
        self.write_summary()
        self.file.close()


# ── Helpers ─────────────────────────────────────────────────

def run(cmd, check=True):
    r = subprocess.run(cmd, shell=True, capture_output=True, text=True)
    if check and r.returncode != 0:
        print(f"FAILED: {cmd}\n{r.stderr}")
        sys.exit(1)
    return r


def systemctl(action):
    run(f"systemctl --user {action} onedrive", check=False)


def btrfs_snapshot(src, dst):
    return run(
        f'sudo btrfs subvolume snapshot "{src}" "{dst}"', check=False
    ).returncode == 0


def btrfs_delete(path):
    run(f'sudo btrfs subvolume delete "{path}"', check=False)


def get_files(root):
    """Return {relative_path_str: absolute_Path} for every file under root."""
    return {str(p.relative_to(root)): p for p in root.rglob("*") if p.is_file()}


def is_remote_wins(rel):
    """True if file is under a REMOTE_WINS prefix. Fallback only — not used when sqlite works."""
    return any(rel == prefix or rel.startswith(prefix + "/") for prefix in REMOTE_WINS)


def save_to_conflicts(src, rel, conflict_dir):
    """Copy remote file to conflicts dir before overwriting or removing it."""
    dst = conflict_dir / rel
    dst.parent.mkdir(parents=True, exist_ok=True)
    shutil.copy2(src, dst)


def cleanup_empty_parents(path, stop_at):
    """Walk up and remove empty dirs left behind after deleting a file."""
    p = path.parent
    while p != stop_at:
        try:
            p.rmdir()   # fails (OSError) if dir not empty — stops the walk
            p = p.parent
        except OSError:
            break


def fmt_mtime(timestamp):
    """Format a unix timestamp as dd.mm for log readability."""
    return datetime.fromtimestamp(timestamp).strftime("%d.%m")


# ── Main ────────────────────────────────────────────────────

def main():
    ts           = datetime.now().strftime("%Y%m%d-%H%M%S")
    snap         = SNAP_DIR / f"pre-resync-{ts}"   # btrfs snapshot path
    conflict_dir = CONFLICT_DIR / ts                # full-copy conflict dir for this run
    log_path     = conflict_dir / "resync.log"      # action log inside conflict dir

    SNAP_DIR.mkdir(parents=True, exist_ok=True)

    # ── determine last sync time ──
    # abraunegg updates items.sqlite3 after every successful sync.
    # Its mtime is the most reliable "last known good sync" timestamp.
    if DB_PATH.exists():
        last_sync     = DB_PATH.stat().st_mtime
        last_sync_str = datetime.fromtimestamp(last_sync).strftime("%d.%m.%Y %H:%M")
    else:
        last_sync     = 0   # 0 = unknown, triggers REMOTE_WINS fallback
        last_sync_str = "UNKNOWN (no sqlite — REMOTE_WINS fallback active)"

    # ── stop monitor ──
    print("==> Stopping onedrive monitor...")
    systemctl("stop")

    # ── btrfs snapshot — captures local state before resync destroys it ──
    print("==> Snapshot local state...")
    if not btrfs_snapshot(SYNC_DIR, snap):
        print("Snapshot failed, aborting.")
        systemctl("start")
        sys.exit(1)

    # ── confirm before the point of no return ──
    print(f"\n  Snapshot:   {snap}")
    print(f"  Last sync:  {last_sync_str}")
    print(f"  Sync dir:   {SYNC_DIR}")

    if input("\nResync now? [y/N] ").strip().lower() != "y":
        print(f"Aborted. Snapshot kept at: {snap}")
        systemctl("start")
        sys.exit(0)

    # ── resync — one API pass, remote state overwrites local ──
    print("==> Resyncing...")
    run("onedrive --sync --resync")

    # ── resolve: compare snapshot (local truth) vs post-resync (remote truth) ──
    print("==> Resolving...\n")
    log = ResyncLog(log_path, last_sync_str)

    local_files  = get_files(snap)      # what you had locally before resync
    remote_files = get_files(SYNC_DIR)  # what remote delivered

    for rel in sorted(set(local_files) | set(remote_files)):
        local  = local_files.get(rel)   # your version (from snapshot)
        remote = remote_files.get(rel)  # remote version (post-resync)

        # ── file exists on both sides ──
        if local and remote:
            if filecmp.cmp(local, remote, shallow=False):
                log.log("unchanged", rel)
            else:
                # local wins — save remote version to conflicts before overwriting
                save_to_conflicts(remote, rel, conflict_dir)
                shutil.copy2(local, remote)
                log.log("overwritten", rel, "local wins, remote → conflicts")

        # ── local only — resync deleted it ──
        elif local and not remote:
            mtime = local.stat().st_mtime

            if last_sync and mtime < last_sync:
                # file predates outage — was synced, then remote deleted it → respect
                log.log("skipped", rel,
                        f"mtime {fmt_mtime(mtime)} < last sync, remote deleted")
            else:
                # file created/modified during outage → restore
                dst = SYNC_DIR / rel
                dst.parent.mkdir(parents=True, exist_ok=True)
                shutil.copy2(local, dst)
                log.log("restored", rel, f"mtime {fmt_mtime(mtime)}")

        # ── remote only — local never had it, or local deleted it ──
        elif remote and not local:
            mtime = remote.stat().st_mtime

            if last_sync and mtime > last_sync:
                # appeared on remote during outage (another device, phone) → keep
                log.log("kept_remote", rel,
                        f"mtime {fmt_mtime(mtime)} > last sync")

            elif last_sync and mtime <= last_sync:
                # old file, unchanged since before outage → local deleted it → respect
                save_to_conflicts(remote, rel, conflict_dir)
                remote.unlink()
                cleanup_empty_parents(remote, SYNC_DIR)
                log.log("removed", rel,
                        f"mtime {fmt_mtime(mtime)} ≤ last sync → conflicts")

            else:
                # no sqlite — can't determine age, use REMOTE_WINS fallback
                if is_remote_wins(rel):
                    log.log("kept_remote", rel, "REMOTE_WINS fallback")
                else:
                    save_to_conflicts(remote, rel, conflict_dir)
                    remote.unlink()
                    cleanup_empty_parents(remote, SYNC_DIR)
                    log.log("removed", rel, "no sqlite → conflicts")

    log.close()

    # ── start monitor — picks up restored files and uploads them ──
    print("\n==> Starting onedrive monitor...")
    systemctl("start")

    # ── prune old snapshots (CoW — low cost, still worth capping) ──
    if SNAP_DIR.exists():
        for old in sorted(SNAP_DIR.glob("pre-resync-*"), reverse=True)[MAX_SNAPSHOTS:]:
            btrfs_delete(old)

    # ── prune old conflict dirs (full copies — can be large) ──
    if CONFLICT_DIR.exists():
        for old in sorted(CONFLICT_DIR.iterdir(), reverse=True)[MAX_CONFLICTS:]:
            shutil.rmtree(old)

    # ── done ──
    print(f"\n==> Done.")
    print(f"  Snapshot:   {snap}")
    print(f"  Conflicts:  {conflict_dir}")
    print(f"  Log:        {log_path}")
    print(f"  Web bin:    https://onedrive.live.com/?id=recycle")
    print(f"\nReview log:")
    print(f"  cat '{log_path}'")
    print(f"  grep '\\[removed\\]' '{log_path}'")
    print(f"  grep '\\[skipped\\]' '{log_path}'")


if __name__ == "__main__":
    main()