Skip to content

Instantly share code, notes, and snippets.

@igortrinidad
Last active February 5, 2024 01:01
Show Gist options
  • Save igortrinidad/9adb01c1c326b42672cbfc7907645e75 to your computer and use it in GitHub Desktop.
Save igortrinidad/9adb01c1c326b42672cbfc7907645e75 to your computer and use it in GitHub Desktop.
How to create AWS Lambda layers using Python pipenv

https://aws.amazon.com/blogs/media/processing-user-generated-content-using-aws-lambda-and-ffmpeg/#:~:text=Adding%20FFmpeg%20layer%20to%20the,and%20select%20S3%20aws%20storage.

# create dir
mkdir my-python-version
cd my-python-version

# Init pipenv with the desired Python version
pipenv install --python 3.12

# Enter pipenv shell
pipenv shell

# Install desired Python packages
pip install [modules name]

#[exit from pipenv shell]

# Go to the Python pipenv folder 
cd [path-to-pipenv-virtualenvs]/my-python-version-[pipenv-hash]/lib/python3.12

# copy site-packages from this python version to a new folder CALLED python - important
cp -r site-packages/* python

# Create the zip
zip -r python-layer-name.zip python

# upload this zip to the Lambda layer new version or create another layer

# Attach this layer (and version) to the Lambda function
  1. Create the Lambda FFmpeg layer Before we start, make sure you are familiar with the terms of FFmpeg license and legal considerations as listed here. In addition, the FFmpeg static build used in this demo is licensed under GPLv3 as mentioned here.

First, we must create a zip package for the Lambda layer that contains the FFmpeg binary. Here is a summary of the steps required to create the FFmpeg package and the Lambda layer. To create the package, you can use an EC2 instance running the same Amazon Machine Image (AMI) used by Lambda (as listed here):

In your working directory, download and unpack the latest static release build of FFmpeg for Linux amd64 from https://johnvansickle.com/ffmpeg/. Instructions are provided in the FAQ of the page and are in the following section for convenience.

wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz

wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz.md5

md5sum -c ffmpeg-release-amd64-static.tar.xz.md5

tar xvf ffmpeg-release-amd64-static.tar.xz

The FFmpeg binary is located in the folder “ffmpeg-4.3.1-amd64-static” (because version 4.3.1 is the latest release available at the time of this post) Create a ZIP package for the Lambda layer as follows:

mkdir -p ffmpeg/bin

cp ffmpeg-4.3.1-amd64-static/ffmpeg ffmpeg/bin/

cd ffmpeg

zip -r ../ffmpeg.zip .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment