Skip to content

Instantly share code, notes, and snippets.

@judzi
Last active May 26, 2024 17:04
Show Gist options
  • Save judzi/981eae8117f80d2bf7bb0a08050889d8 to your computer and use it in GitHub Desktop.
Save judzi/981eae8117f80d2bf7bb0a08050889d8 to your computer and use it in GitHub Desktop.
Google Takeout Merge Tool: Unzips and merges all your exported data into a single folder
#!/usr/bin/env bash
destDir="../Takeout-merge"
mkdir -p $destDir
# Merges the different unzipped Takeout folders into one
for p in $(seq -f "%03g" 1 ${1:-1}); do
if [ -f $p/DONE ]; then
echo "Skipping $p/Takeout ... "
continue
fi
echo "Merging $p/Takeout into {$destDir} ... "
# t = preserve modification times
# q = quiet, suppress non-error messages
# r = recursive on folders
# X = preserved extended attributes
# X = skip files which are newer in dest than in src
# h = output numbers in a human-readable format
rsync -rtXhu --info=progress2 --safe-links --remove-source-files $p/Takeout {$destDir}
touch "$p/DONE"
echo "DONE merging $p"
done
echo "ALL MERGED!"
#!/usr/bin/env bash
# Unzips all Takeout archive files (living in the same dir)
archiveTimestamp=${2:-"20000101T100000Z"}
for p in $(seq -f "%03g" 1 ${1:-1}); do
f="takeout-${archiveTimestamp}-$p.zip"
if [ ! -f "$f" ]; then
echo "Archive not found: $f. Skipping ... "
continue
fi
mkdir -p $p
cd $p
echo "Unzipping takeout-${archiveTimestamp}-$p.zip ..."
unzip "../takeout-${archiveTimestamp}-$p.zip"
echo "DONE unzipping $p"
cd ..
done
echo "ALL UNZIPPED!"
@judzi
Copy link
Author

judzi commented Sep 28, 2020

Usage:

Given a directory with this file pattern:

./takeout-20181016T122422Z-001.tgz
./takeout-20181016T122422Z-002.tgz
./takeout-20181016T122422Z-003.tgz
Then we call these commands:

./takeout-unzip.sh 3 20181016T122422Z
./takeout-merge.sh 3
And everything will be merged under ./../Takeout-merge

@chabala
Copy link

chabala commented Oct 27, 2020

Why would you need a script (or in your case two) for this? Use brace expansion, cat the TGZs, and extract:
cat takeout-20201023T123551Z-{001..011}.tgz | tar xzivf -

@nitishd
Copy link

nitishd commented Jun 6, 2021

They were zip files.

Just this works:

unzip '*.zip' -d complete-takeout

@amhasbi
Copy link

amhasbi commented Jul 16, 2021

Usage:

Given a directory with this file pattern:

./takeout-20181016T122422Z-001.tgz
./takeout-20181016T122422Z-002.tgz
./takeout-20181016T122422Z-003.tgz
Then we call these commands:

./takeout-unzip.sh 3 20181016T122422Z
./takeout-merge.sh 3
And everything will be merged under ./../Takeout-merge

Dear, author. I am completely lost, zero, nothing in programming. I just need to merge and extract all the data from my drive and I found this page which I believe would solve my issue. But I have no idea how to do this. Can you elaborate it a little bit more for me in terms what tools to use, how to operate, etc. Any help will be so much appreciated. Thanks in advance!

@MarkusKress
Copy link

There is an Bug in rsync line: {$destDir} -> ${destDir}

@chabala
Copy link

chabala commented Feb 10, 2022

@schplorg
Copy link

Why would you need a script (or in your case two) for this? Use brace expansion, cat the TGZs, and extract: cat takeout-20201023T123551Z-{001..011}.tgz | tar xzivf -

This worked like a charm, even on Windows using Git bash 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment