Created
May 26, 2021 07:56
-
-
Save akshat-khare/13421b911db8f80b961d83415b18db47 to your computer and use it in GitHub Desktop.
Facebook data to google drive
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "nbformat": 4, | |
| "nbformat_minor": 0, | |
| "metadata": { | |
| "colab": { | |
| "name": "Facebook data to google drive", | |
| "provenance": [], | |
| "collapsed_sections": [], | |
| "toc_visible": true, | |
| "mount_file_id": "1_9DZqsFowB90w8DrKZZiO1y2XTTcryBP", | |
| "authorship_tag": "ABX9TyPcyqFRxx6Yq8yoPjg/qK7+", | |
| "include_colab_link": true | |
| }, | |
| "kernelspec": { | |
| "name": "python3", | |
| "display_name": "Python 3" | |
| }, | |
| "language_info": { | |
| "name": "python" | |
| } | |
| }, | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "view-in-github", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "<a href=\"https://colab.research.google.com/gist/akshat-khare/13421b911db8f80b961d83415b18db47/facebook-data-to-google-drive.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "DKzAIMTB95Vk" | |
| }, | |
| "source": [ | |
| "# How to download all of the facebook data you have to google drive/personal pc . . \n", | |
| "Facebook to google colab to google drive . . /\n", | |
| "\n", | |
| "This process is quite simple and won't take more than 20 minutes of your time . . First of open facebook and go to dropdown > Settings and Privacy > Settings." | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "6VFXjV8I-1a6" | |
| }, | |
| "source": [ | |
| "" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "3eOfepJ__SL2" | |
| }, | |
| "source": [ | |
| "Click on Your Facebook Information > Download Your Information > Request Copy > Create File. \n", | |
| "Your file will be created in 5-6 hours on facebook. When it is done you will receive a mail on your registered id. Then go to Available Copies which is just right of Request Copy. There you will see your data. " | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "tZJ05OBRAKmJ" | |
| }, | |
| "source": [ | |
| "" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "yWTpYsPVAP5k" | |
| }, | |
| "source": [ | |
| "Now comes the tricky part. Open Network tab on your browser control. You can do this by pressing Fn + F12 together. If not, Google 'How to open network tab in browser'. You will need to refresh the page for network tab to start logging traffic of page. \n", | |
| "After reloading, click on Download/Download Again. You might need to authenticate again. See the Name of query which is being run. \n", | |
| "When a download window appears press cancel and immediately see the Name of some query like `file.php?r=100..`. Right click the query > Copy > Copy as curl.\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "D3F6_wDiB5RF" | |
| }, | |
| "source": [ | |
| "" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "tttJP3phCESC" | |
| }, | |
| "source": [ | |
| "Paste it on ipython with `!` in start and `--output facebook1.zip` in the end like below. I have omitted part of my query which might expose my data." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| }, | |
| "id": "PTsBJkNHxQoa", | |
| "outputId": "40f70a17-5db2-4274-9f1e-425610ac62ab" | |
| }, | |
| "source": [ | |
| "!curl 'https://bigzipfiles.facebook.com/p/dl/download/file.php?r=1000....' \\\n", | |
| " -H 'authority: bigzipfiles.facebook.com' \\\n", | |
| " -H 'sec-ch-ua: \" Not A;Brand\";v=\"99\", \"Chromium\";v=\"90\", \"Google Chrome\";v=\"90\"' \\\n", | |
| " -H 'sec-ch-ua-mobile: ?0' \\\n", | |
| " -H 'upgrade-insecure-requests: 1' \\\n", | |
| " -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36' \\\n", | |
| " -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \\\n", | |
| " -H 'sec-fetch-site: same-site' \\\n", | |
| " -H 'sec-fetch-mode: navigate' \\\n", | |
| " -H 'sec-fetch-user: ?1' \\\n", | |
| " -H 'sec-fetch-dest: document' \\\n", | |
| " -H 'referer: https://www.facebook.com/' \\\n", | |
| " -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \\\n", | |
| " -H 'cookie: datr=2T...' \\\n", | |
| " --compressed --output facebook1.zip" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| " % Total % Received % Xferd Average Speed Time Time Time Current\n", | |
| " Dload Upload Total Spent Left Speed\n", | |
| "100 1923M 100 1923M 0 0 26.3M 0 0:01:12 0:01:12 --:--:-- 29.9M\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| }, | |
| "id": "SXi3XgY8xXdr", | |
| "outputId": "5c334147-2b06-41ff-e272-8e085b117db1" | |
| }, | |
| "source": [ | |
| "!ls -lhtr" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "total 1.9G\n", | |
| "drwxr-xr-x 1 root root 4.0K May 6 13:44 sample_data\n", | |
| "-rw-r--r-- 1 root root 1.9G May 26 03:45 facebook1.zip\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "Wa4Z6ORvCftu" | |
| }, | |
| "source": [ | |
| "You can see we have first data file downloaded with us. Do the same for all files in facebook dashboard." | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| }, | |
| "id": "av2HxwiMxtn_", | |
| "outputId": "d1e43dc0-630a-4844-b8e1-704209462270" | |
| }, | |
| "source": [ | |
| "!curl 'https://bigzipfiles.facebook.com/p/dl/download/file.php?r=100...' \\\n", | |
| " -H 'authority: bigzipfiles.facebook.com' \\\n", | |
| " -H 'sec-ch-ua: \" Not A;Brand\";v=\"99\", \"Chromium\";v=\"90\", \"Google Chrome\";v=\"90\"' \\\n", | |
| " -H 'sec-ch-ua-mobile: ?0' \\\n", | |
| " -H 'upgrade-insecure-requests: 1' \\\n", | |
| " -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36' \\\n", | |
| " -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \\\n", | |
| " -H 'sec-fetch-site: same-site' \\\n", | |
| " -H 'sec-fetch-mode: navigate' \\\n", | |
| " -H 'sec-fetch-user: ?1' \\\n", | |
| " -H 'sec-fetch-dest: document' \\\n", | |
| " -H 'referer: https://www.facebook.com/' \\\n", | |
| " -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \\\n", | |
| " -H 'cookie: datr=2TN....' \\\n", | |
| " --compressed --output facebook2.zip" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| " % Total % Received % Xferd Average Speed Time Time Time Current\n", | |
| " Dload Upload Total Spent Left Speed\n", | |
| "100 3555M 100 3555M 0 0 12.3M 0 0:04:48 0:04:48 --:--:-- 12.2M\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| }, | |
| "id": "8V5yRQsbx2_i", | |
| "outputId": "0ea5c4b9-3a42-44e7-b96e-e9a904f5ac82" | |
| }, | |
| "source": [ | |
| "!ls -lhtr" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "total 5.4G\n", | |
| "drwxr-xr-x 1 root root 4.0K May 6 13:44 sample_data\n", | |
| "-rw-r--r-- 1 root root 1.9G May 26 03:45 facebook1.zip\n", | |
| "-rw-r--r-- 1 root root 3.5G May 26 03:50 facebook2.zip\n", | |
| "drwx------ 6 root root 4.0K May 26 03:50 drive\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| }, | |
| "id": "LU-D6j0xzBEE", | |
| "outputId": "1367adf3-e92f-4e8a-ccb0-0d466a8a9e28" | |
| }, | |
| "source": [ | |
| "!curl 'https://bigzipfiles.facebook.com/p/dl/download/file.php?r=100..' \\\n", | |
| " -H 'authority: bigzipfiles.facebook.com' \\\n", | |
| " -H 'sec-ch-ua: \" Not A;Brand\";v=\"99\", \"Chromium\";v=\"90\", \"Google Chrome\";v=\"90\"' \\\n", | |
| " -H 'sec-ch-ua-mobile: ?0' \\\n", | |
| " -H 'upgrade-insecure-requests: 1' \\\n", | |
| " -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36' \\\n", | |
| " -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \\\n", | |
| " -H 'sec-fetch-site: same-site' \\\n", | |
| " -H 'sec-fetch-mode: navigate' \\\n", | |
| " -H 'sec-fetch-user: ?1' \\\n", | |
| " -H 'sec-fetch-dest: document' \\\n", | |
| " -H 'referer: https://www.facebook.com/' \\\n", | |
| " -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \\\n", | |
| " -H 'cookie: datr=2TN..' \\\n", | |
| " --compressed --output facebook3.zip" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| " % Total % Received % Xferd Average Speed Time Time Time Current\n", | |
| " Dload Upload Total Spent Left Speed\n", | |
| "100 4997M 100 4997M 0 0 28.4M 0 0:02:55 0:02:55 --:--:-- 40.9M\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "SEXRDR9eC6W2" | |
| }, | |
| "source": [ | |
| "Now that all the data is downloaded in colab runtime, *it is important to also upload it in Google drive because we will lose the data when colab shuts down.* You can just chose to download it in your pc by clicking 3 dots besides the file in explorer on right . . \n", | |
| "First you need to mount the drive in colab. Open files tab from left and press Mount Drive" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "QpjfjdysDW5f" | |
| }, | |
| "source": [ | |
| "" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "oAMCytuRDZdQ" | |
| }, | |
| "source": [ | |
| "Now create the relevant folders like I made named `facebook_backup_mummy` for my mother. Also a subfolder like `1` for " | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "aOM9zD5gzI-_" | |
| }, | |
| "source": [ | |
| "!cp facebook1.zip facebook2.zip facebook3.zip /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1/" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| }, | |
| "id": "1i0QUMM5z-XD", | |
| "outputId": "6dbd1cb1-b3fc-427b-d27f-a3f67da56c62" | |
| }, | |
| "source": [ | |
| "!ls -lhtr /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "total 11G\n", | |
| "-rw------- 1 root root 1.9G May 26 03:55 facebook1.zip\n", | |
| "-rw------- 1 root root 3.5G May 26 03:57 facebook2.zip\n", | |
| "-rw------- 1 root root 4.9G May 26 04:00 facebook3.zip\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "colab": { | |
| "base_uri": "https://localhost:8080/" | |
| }, | |
| "id": "Z-5Wv0Zt1fIn", | |
| "outputId": "16851afa-69d7-49dd-bedb-cc8050183f03" | |
| }, | |
| "source": [ | |
| "!ls -lhtr /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1" | |
| ], | |
| "execution_count": null, | |
| "outputs": [ | |
| { | |
| "output_type": "stream", | |
| "text": [ | |
| "total 11G\n", | |
| "-rw------- 1 root root 1.9G May 26 03:55 facebook1.zip\n", | |
| "-rw------- 1 root root 3.5G May 26 03:57 facebook2.zip\n", | |
| "-rw------- 1 root root 4.9G May 26 04:00 facebook3.zip\n" | |
| ], | |
| "name": "stdout" | |
| } | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "iLcc52QL0e-T" | |
| }, | |
| "source": [ | |
| "!unzip /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1/facebook1.zip -d /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1_folder/facebook1" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "rxJWbkYk1j96" | |
| }, | |
| "source": [ | |
| "!unzip /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1/facebook2.zip -d /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1_folder/facebook2" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "Y7p-PbnA3ABk" | |
| }, | |
| "source": [ | |
| "!unzip /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1/facebook3.zip -d /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1_folder/facebook3" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "metadata": { | |
| "id": "hawvYQMh3Fto" | |
| }, | |
| "source": [ | |
| "!du -h /content/drive/MyDrive/Colab_Notebooks/facebook_backup_mummy/1_folder" | |
| ], | |
| "execution_count": null, | |
| "outputs": [] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "xFMwUPFZqn0f" | |
| }, | |
| "source": [ | |
| "Around 11GB of my mother's most prized facebook data is now backup-ed which she can see in her google drive :)" | |
| ] | |
| } | |
| ] | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment