Skip to content

Instantly share code, notes, and snippets.

@Asparagirl
Last active November 11, 2022 06:09
Show Gist options
  • Save Asparagirl/e3ee274e4df49230875c880255819d95 to your computer and use it in GitHub Desktop.
Save Asparagirl/e3ee274e4df49230875c880255819d95 to your computer and use it in GitHub Desktop.
How to generate a Twitter user's unique tweet URL's, and then feed them into ArchiveBot to be saved

Set up Tweep

  1. Download the file tweep.py from GitHub - https://github.com/haccer/tweep

  2. Put it in a brand new folder. Let's call the folder "Tweep". So the full path here would be, as an example, /Users/asparagirl/Desktop/Tweep

  3. Add a folder inside of that one called tmp. So the full path here would be, as an example, /Users/asparagirl/Desktop/Tweep/tmp

  4. Edit tweep.py slightly to add logging and stop it from getting images from tweets. The top of the file should be edited to look like this:

#!/usr/bin/env python
import logging
logging.basicConfig(level=logging.DEBUG)
from bs4 import BeautifulSoup
from time import gmtime, strftime
# from PIL import Image
from io import BytesIO
import argparse

...etc.

  1. Install tweep's dependencies:
  • pip install beautifulsoup4
  • pip install requests
  • pip install lxml

(You may need to do sudo pip instead of just pip, depending on your system.)

Run Tweep to start collecting Tweets

  1. Run
python tweep.py -u TWITTER-USERNAME-HERE > /Users/asparagirl/Desktop/Tweep/tmp/TWITTER-USERNAME-HERE
  1. You should see the screen filling with log data. The tweets will be stored in a text file called TWITTER-USERNAME-HERE inside the tmp sub-directory.

  2. Now we need to parse that text file and just get the full Twitter status URL's, and discard the actual text of the tweets. Run:

awk '{print "https://twitter.com/" substr($5, 2, length($5) - 2) "/status/" $1}' /Users/asparagirl/Desktop/Tweep/tmp/TWITTER-USERNAME-HERE | pbcopy -selection clipboard
  1. This will copy to your clipboard the list of tweets!

  2. Paste that data to pastebin or whatever you need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment