haranjackson’s gists

haranjackson / python_function_name_duplicates.sh

Created May 26, 2020 23:20

Shell script for finding Python function name duplicates

	find . -not -path '/\.' -type f -name "*.py" \| # find non-hidden python files
	xargs egrep -o 'def [A-Za-z0-9_]{1,}' \| # find function def lines
	egrep -v 'def __[a-z]{1,}__' \| # ignore __init__, __repr__,...
	awk -F "def " '{print $NF}' \| # take function names
	sort \| # sort list of function names
	uniq -c \| # count occurrences of each
	sort # sort again by count

haranjackson / authAtEdge.yml

Last active April 5, 2020 00:33

CloudFormation template for quickly creating a password-protected website. Put your website's source in folder "src" next to authAtEdge.yml and deploy.sh. Based on: github.com/aws-samples/cloudfront-authorization-at-edge.

	AWSTemplateFormatVersion: 2010-09-09
	Transform: AWS::Serverless-2016-10-31

	Parameters:
	DomainName:
	Type: String
	Description: Domain name of the website

	HostedZone:
	Type: String

haranjackson / deploy_wiki.sh

Created February 5, 2020 16:53

Deploys BookStack wiki on AWS EC2

	STACK= # give the stack a name
	REGION= # choose your region
	DOMAIN= # a Route53 domain - wiki will be hosted at wiki.[DOMAIN]

	aws cloudformation deploy \
	--template-file wiki.yaml \
	--stack-name $STACK \
	--region $REGION \
	--parameter-overrides Domain=$DOMAIN

haranjackson / sumy_lambda_layer.sh

Created November 29, 2019 00:20

Deploys Python sumy library to an AWS Lambda layer. You can specify the region, library version, and runtime. You must specify a temporary S3 bucket to use.

	# must set environment variable NLTK_DATA=/opt/python in the lambda function

	REGION=eu-west-1
	RUNTIME=python3.7
	BUCKET=

	VER=0.8.1
	NUMPY_VER=1.17.4

	OUT_DIR=/out/build/sumy/python

haranjackson / pypi_deployment_workflow.yml

Last active November 26, 2019 02:52

GitHub workflow for deploying a Python package to PyPI. The package can contain a CMake build step. Wheels are built for Linux, MacOS, and Windows, for Python versions 3.6-3.8. You must first create a GitHub secret called 'pypi_password' containing your PyPI API key.

	name: CI

	on: [push]

	jobs:
	nonlinux-build:
	strategy:
	fail-fast: false
	matrix:
	os: [macOS-latest, windows-latest]

haranjackson / pdfminer_lambda_layer.sh

Created August 30, 2019 12:18

Deploys Python PDFMiner library to an AWS Lambda layer (removing unnecessary test files to reduce size). You can specify the region, library version, and runtime.

	REGION=eu-west-1
	VER=20181108
	RUNTIME=python3.7

	docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \
	pip install pdfminer.six==$VER -t /out/build/pdfminer/python

	pushd build/pdfminer
	rm -rf python/Crypto/SelfTest/
	zip -r ../../pdfminer.zip python/

haranjackson / chrome_headless.py

Last active November 4, 2024 19:45

Deploys the Python Selenium library and Chrome Headless to an AWS Lambda layer. You can specify the region, library version, and runtime. An example Lambda function is given.

	from selenium.webdriver import Chrome
	from selenium.webdriver.chrome.options import Options

	options = Options()

	options.binary_location = '/opt/headless-chromium'
	options.add_argument('--headless')
	options.add_argument('--no-sandbox')
	options.add_argument('--start-maximized')
	options.add_argument('--start-fullscreen')

haranjackson / spacy_lambda_layer.sh

Created August 15, 2019 14:33

Deploys Python Spacy library to an AWS Lambda layer. You can specify the region, library version, language, language model, and runtime.

	REGION=eu-west-1
	VER=2.1.8
	LANG=en
	MODEL=en_core_web_sm-2.1.0
	RUNTIME=python3.7

	MODEL_URL=https://github.com/explosion/spacy-models/releases/download/$MODEL/$MODEL.tar.gz

	docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \
	pip install spacy==$VER $MODEL_URL -t /out/build/spacy/python

haranjackson / scrapy_lambda_layer.sh

Last active May 7, 2022 09:24

Deploys Python Scrapy library to an AWS Lambda layer. You can specify the region, library version, and runtime.

	REGION=eu-west-1
	VER=1.7.3
	RUNTIME=python3.7

	docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \
	pip install scrapy==$VER -t /out/build/scrapy/python

	cd build/scrapy
	zip -r ../../scrapy.zip python/
	cd ../..

haranjackson / api_deployment.yaml

Created August 11, 2019 16:48

An AWS CloudFormation template for a 2-stage (prod/dev) deployment on an API using API Gateway, with optional API key

	AWSTemplateFormatVersion: 2010-09-09

	Parameters:

	ApiId:
	Type: String

	UsagePlanName:
	Type: String
	Description: >-

	find . -not -path '/\.' -type f -name "*.py" \| # find non-hidden python files
	xargs egrep -o 'def [A-Za-z0-9_]{1,}' \| # find function def lines
	egrep -v 'def __[a-z]{1,}__' \| # ignore __init__, __repr__,...
	awk -F "def " '{print $NF}' \| # take function names
	sort \| # sort list of function names
	uniq -c \| # count occurrences of each
	sort # sort again by count

Hari Jackson haranjackson