gif-from-tweet.md

gif-from-tweet

There are so many great GIFs out there and I want to have copies of them. Twitter makes that harder than it should be by converting them to MP4 and not providing access to the source material. To make it easier, I made a bash pipeline that takes a tweet URL and a filename, extracts the MP4 from that tweet and uses ffmpeg to convert back to GIF.

Dependencies

• ffmpeg
  • macOS: brew install ffmpeg
  • Ubuntu/Debian: apt install ffmpeg

Install

Stick this in your ~/.profile:

video-url-from-tweet() {
    if [ "$1" ]; then
        url=$1
    else
        echo "Must provide a url"
        return 1
    fi

    curl --silent $url |\
        # should find the <meta> tag with content="<thumbnail url>"
        (grep -m1 "tweet_video_thumb" ||\
          echo "Could not find video" && return 1) |\

        # from: <meta property="og:image" content="https://pbs.twimg.com/tweet_video_thumb/xxxxxxxxxx.jpg">
        # to: https://pbs.twimg.com/tweet_video_thumb/xxxxxxxxxx.jpg
        cut -d '"' -f 4 |\

        # from: https://pbs.twimg.com/tweet_video_thumb/xxxxxxxxxx.jpg
        # to: https://video.twimg.com/tweet_video/xxxxxxxxxx.mp4
        sed 's/.jpg/.mp4/g' |\
        sed 's/pbs.twimg.com\/tweet_video_thumb/video.twimg.com\/tweet_video/g'
}
video-from-tweet() {
    if [ "$1" ]; then
        url=$1
    else
        echo "Must provide a url"
        return 1
    fi
    curl $(video-url-from-tweet $url)
}
video-to-gif() {
    # derived from https://engineering.giphy.com/how-to-make-gifs-with-ffmpeg/
    if [ "$2" ]; then
        input=$1
        output=$2
    else
        echo "Must provide an input file and output file"
        return 1
    fi

    ffmpeg -i $input \
           -filter_complex "[0:v] split [a][b];[a] palettegen [p];[b][p] paletteuse" \
           -f gif \
           $output
}
gif-from-tweet() {
    if [ "$2" ]; then
        url=$1
        output=$2
    else
        echo "Must provide a url and an output filename"
        return 1
    fi
    video-from-tweet $url | video-to-gif - $output
}

Usage

• video-url-from-tweet <url>: takes a tweet URL and returns the MP4 embedded in that tweet, or fails if no video is found.
• video-from-tweet <url>: returns the raw data of the video that is embedded in the tweet
• video-to-gif <input> <output>: converts a video to a GIF
• gif-from-tweet <url> <output>: takes a tweet URL and an output filename and saves the MP4 embedded in that tweet as a GIF.

Example

$ video-url-from-tweet https://twitter.com/tsunamino/status/1003318804619804672
https://video.twimg.com/tweet_video/DeyBINOUwAAbuif.mp4

# creates `wink.mp4'
$ video-from-tweet https://twitter.com/tsunamino/status/1003318804619804672 > wink.mp4

# creates `wink.gif' from `wink.mp4'
$ video-to-gif wink.mp4 wink.gif
<...a bunch of ffmpeg output...>

# or use this, which pipelines the above and doesn't create intermediate MP4
$ gif-from-tweet https://twitter.com/tsunamino/status/1003318804619804672 wink.gif

For the Ubuntu/Debian heads; running apt install ffmpeg will do just fine!

@jalcine thanks, updated the instructions!

For reference, in Fedora, this should be added to ~/.bash_rc. One can then re-read it without logging out and back in by running
source ~/.bashrc

Nice! There is also https://github.com/rg3/youtube-dl which supports almost every site with media (including Twitter, Soundcloud, youtube, vimeo etc). Similar usage -

youtube-dl https://twitter.com/tsunamino/status/1003318804619804672

@eugene1g Maybe I'm missing something, but all pointing youtube-dl at a tweet nets me is "ERROR: No video formats found."

This is very neat!

As FYI If you want to run it under ubuntu for windows, you need to install ffmpeg differently. See here: https://www.reddit.com/r/Windows10/comments/5006is/want_to_install_ffmpeg_under_bash_for_windows/

sudo add-apt-repository ppa:mc3man/trusty-media
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install ffmpeg

Note you can install ffmpeg for windows from https://ffmpeg.zeranoe.com/builds/ then use this gist with Git Bash or Cygwin

I don’t know if it’s some sort of weird local configuration on my part, but I had to put the routines into .bash_profile rather than .profile for bash to be able to use them. Thanks for this!

Hey @brianloveswords Amazing stuff :D ! I found that this doesn't work for some tweets which have the og:image property in a different way. For example, https://twitter.com/knjiminie/status/1005574807914729472, the value is https://pbs.twimg.com/ext_tw_video_thumb/1005574542151028736/pu/img/-WtqAkKG4PmvMIzz.jpg. Any idea how to handle this case?

@aulisius it’s because that media content isn’t a gif but rather a video. With the sed command replacing the extension from jpg to mp4, it looks like Twitter doesn’t store pure videos in the same way. To get things to work with a true video tweet to gif you’d have to figure how these are stored in the Twitter backend to get at the video. Viewing the source of the tweet could help you figure it out.

https://pbs.twimg.com/ext_tw_video_thumb/1005574542151028736/pu/img/-WtqAkKG4PmvMIzz.mp4 Results in a 404/unplayable video

Tried out youtube-dl as suggested by @eugene1g and does the trick:-

tweet-to-gif() {
  youtube-dl -o - $1 | ffmpeg -i pipe: \
    -filter_complex "[0:v] split [a][b];[a] palettegen [p];[b][p] paletteuse" \
    -hide_banner -loglevel panic \
    -f gif ${1##*/}.gif
}

---

tweet-to-gif https://twitter.com/garrett_white/status/1182475631503785984

@eugene1g:

Thank you for pointing this out.

@gaving:

Thanks for the excellent script! It works perfectly.

As pointed out by meyerweb, though it was not mentioned why, this must be added to ~/.bash_profile in OS X. That said, it does not work at all.

video-url-from-tweet https://twitter.com/tsunamino/status/1003318804619804672
Could not find video

All other commands fail similarly.

Thank gawd for comments. As eugene1g pointed out, youtube-dl can download the video of the animated GIF.

And as gaving pointed out, you can directly download the video and convert it to animated GIF with his brilliant script added to your profile file:

tweet-to-gif() {
  youtube-dl -o - $1 | ffmpeg -i pipe: \
	-filter_complex "[0:v] split [a][b];[a] palettegen [p];[b][p] paletteuse" \
	-hide_banner -loglevel panic \
	-f gif ${1##*/}.gif
}

Note that both the twitter thread URL and the URL you get from right-clicking the GIF and clicking Copy Gif Address both work.

e.g.,
Thread URL: https://twitter.com/Astropartigirl/status/1479837621337657345
GIF URL: https://twitter.com/i/status/1479837621337657345

I deleted the original script in my ~/.bash_profile and added this one.