Skip to content

Instantly share code, notes, and snippets.

@Sillium
Forked from haranjackson/chrome_headless.py
Last active December 8, 2021 13:11
Show Gist options
  • Save Sillium/bee21eff357a34da11f998df682c7104 to your computer and use it in GitHub Desktop.
Save Sillium/bee21eff357a34da11f998df682c7104 to your computer and use it in GitHub Desktop.
Deploys the Python Selenium library and Chrome Headless to an AWS Lambda layer. You can specify the region, library version, and runtime. An example Lambda function is given.

After installation, run this:

aws lambda add-layer-version-permission --layer-name ChromeHeadless --statement-id xaccount --action lambda:GetLayerVersion  --principal "*" --version-number X --output text
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location = '/opt/headless-chromium'
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--start-maximized')
options.add_argument('--start-fullscreen')
options.add_argument('--single-process')
options.add_argument('--disable-dev-shm-usage')
driver = Chrome('/opt/chromedriver', options=options)
#!/bin/bash
REGION=eu-central-1
RUNTIME=python3.7
BUCKET=sillium-layers-miscellaneous
SELENIUM_VER=3.141.0
CHROME_BINARY_VER=v1.0.0-55 # based on Chromium 69.0.3497.81
CHROMEDRIVER_VER=2.43 # supports Chrome v69-71
OUT_DIR=/out/build/chrome_headless/python/lib/$RUNTIME/site-packages
docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \
pip install selenium==$SELENIUM_VER -t $OUT_DIR
cp chrome_headless.py build/chrome_headless/python/chrome_headless.py
pushd build/chrome_headless
DRIVER_URL=https://chromedriver.storage.googleapis.com/$CHROMEDRIVER_VER/chromedriver_linux64.zip
curl -SL $DRIVER_URL >chromedriver.zip
unzip chromedriver.zip
rm chromedriver.zip
# download chrome binary
CHROME_URL=https://github.com/adieuadieu/serverless-chrome/releases/download/$CHROME_BINARY_VER/stable-headless-chromium-amazonlinux-2017-03.zip
curl -SL $CHROME_URL >headless-chromium.zip
unzip headless-chromium.zip
rm headless-chromium.zip
zip -r ../../chrome_headless.zip *
popd
aws s3 cp chrome_headless.zip s3://$BUCKET/chrome_headless.zip
aws lambda publish-layer-version \
--layer-name ChromeHeadless \
--region $REGION \
--content S3Bucket=$BUCKET,S3Key=chrome_headless.zip \
--compatible-runtimes $RUNTIME
rm -rf build *.zip
from chrome_headless import driver
def lambda_handler(event, context):
driver.get(event['url'])
return driver.page_source
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment