Example use of spleeter (Python+Tensorflow audio-extraction library and ffmpeg

This is a guide that basically combines protolium's very helpful ffmpeg cheatsheet with the spleeter library.

Here's a tweet thread that shows a video snippet, with separate bass, vocals, and drums track:

(this guide was tested on ffmpeg 4.1.4)

Lots of ways to do this. If you're on a Mac, I suggest using Homebrew:

brew install ffmpeg

Install (and isolate) using conda:

git clone https://github.com/Deezer/spleeter
conda env create -f spleeter/conda/spleeter-cpu.yaml
conda activate spleeter-cpu

You can also do pip install spleeter but you might run into more than a few library conflicts with your existing Python environment...

Given a video.mp4 file, here's how to use ffmpeg to extract the audio into a separate file, audio.wav:

ffmpeg -i video.mp4 -vn -f wav audio.wav

The following command invokes spleeter to split audio.wav into 5 separate tracks, into a subdirectory called myoutput/audio

spleeter separate -i audio.wav -p spleeter:5stems -o myoutput

The generated files:

myoutput/audio/
├── bass.wav
├── drums.wav
├── other.wav
├── piano.wav
└── vocals.wav

Basically, create a silent version of the video, and then merge it with one of the given track extracts.

This creates a copy of the video, sans audio track, named silent-video.mp4 from video.mp4

ffmpeg -i video.mp4 -codec copy -an silent-video.mp4

Here's an example that creates drums-video.mp4: the video, with just the drums track:

ffmpeg -i silent-video.mp4 -i myoutput/audio/drums.wav -c:v copy -c:a aac drums-video.mp4

If you want to snippet any media file, here's an example using ffmpeg that cuts a 12-second snippet, beginning at the 30th second:

ffmpeg -i video.mp4 -ss 30 -t 12  video-snippet.mp4

dannguyen/spleeter-and-ffmpeg-quick-tutorial.md