-
-
Save nemani/defdde356b6678352bcd4af69b7fe529 to your computer and use it in GitHub Desktop.
# Parallelly download all aws-lambda functions | |
# Assumes you have ran `aws configure` and have output-mode as "text" | |
# Works with "aws-cli/1.16.72 Python/3.6.7 Linux/4.15.0-42-generic botocore/1.12.62" | |
download_code () { | |
local OUTPUT=$1 | |
aws lambda get-function --function-name $OUTPUT --query 'Code.Location' | xargs wget -O ./lambda_functions/$OUTPUT.zip | |
} | |
mkdir -p lambda_functions | |
for run in $(aws lambda list-functions | cut -f 6 | xargs); | |
do | |
download_code "$run" & | |
done | |
echo "Completed Downloading all the Lamdba Functions!" | |
# If you want to download only ones with specific prefix | |
# https://github.com/sambhajis-gdb/download_all_lambda_function/blob/master/get_lambda_with_prefix.sh | |
# Credits to https://stackoverflow.com/users/6908140/sambhaji-sawant for the |
It crashed my 64GB RAM mac, Twice :/
I can verify that the original version does that on Macs. @handeglc removed the & to not run in background. I did the same and also added a loop to capture all regions. It could be improved even more of course. Thanks @nemani for posting this gist!
# Downloads all aws-lambda functions to a subdirectory
# Assumes you have a role that has at least read access to lambda.
# Credits to https://gist.github.com/nemani/defdde356b6678352bcd4af69b7fe529
# trap ctrl-c and call ctrl_c()
trap ctrl_c INT
function ctrl_c() {
echo "** CTRL-C Detected - aborting..."
exit -1
}
download_region(){
REGION=$1
echo Searching region: $REGION
for code in $(aws lambda list-functions --region $REGION --query 'Functions[].FunctionName' --output text);
do
download_path=./lambda_functions/$REGION/$code
mkdir -p $download_path
aws lambda get-function --function-name $code --region $REGION --query 'Code.Location' | xargs wget --quiet -O $download_path/$code.zip
unzip -o -f -d $download_path $download_path/$code.zip
done
}
# search all regions
for region in $(aws ec2 describe-regions --query "Regions[].{Name:RegionName}" --output text | sort -r); do
download_region $region
done
echo "Completed Downloading all the Lamdba Functions!"
Thanks for sharing, the code didn't work as stated in some previous replies I believe because of the filtering criteria to get the lambda function name.
AWS CLI is returning a JSON format which can be better explored using jq
If the above code didn't work for you too, try my implementation of the fix here.
@abhay330 thanks man! Worked very well
very nice collaboration. I extended it in a fork https://gist.github.com/bf4/825009c3e49bf53259e8c18c4c38f190
#!/bin/bash
#
# Usage:
# ./download_all.sh
# ./download_all.sh download us-east-1 my-function
# ./download_all.sh help
#
# Downloads all aws-lambda functions to a subdirectory
# Assumes you have a role that has at least read access to lambda.
# Credits to https://gist.github.com/nemani/defdde356b6678352bcd4af69b7fe529
set -euo pipefail
IFS=$'\n\t'
# Influenced by https://dev.to/thiht/shell-scripts-matter
abort=0
show_help() {
abort=${abort:0}
sed -ne '/^#/!q;s/.\{1,2\}//;1d;p' < "$0" >&$((abort+1))
exit $abort
}
# shellcheck disable=SC2034
export AWS_PROFILE="${AWS_PROFILE:-your-profile-here}"
readonly DOWNLOAD_PATH_BASE="./lambda_functions"
# trap ctrl-c and call ctrl_c()
trap ctrl_c INT
ctrl_c() {
echo "** CTRL-C Detected - aborting..."
exit 1
}
get_function_names() {
local region; region="$1"
aws lambda list-functions --region "$region" --query 'Functions[].FunctionName' --output text
}
get_function() {
local region; region="$1"
local function_name; function_name="$2"
local download_path
download_path="${DOWNLOAD_PATH_BASE}/$region/$function_name"
mkdir -p "$download_path"
aws lambda get-function --function-name "$function_name" --region "$region" --query 'Code.Location' \
| xargs wget --quiet -O "$download_path/$function_name.zip"
unzip -o -f -d "$download_path" "$download_path/$function_name.zip"
}
download_region(){
local region; region="$1"
echo "searching region: $region"
for function_name in $(get_function_names "$region"); do
get_function "$region" "$function_name"
done
}
get_regions(){
aws ec2 describe-regions --query "Regions[].{Name:RegionName}" --output text | sort -r
}
main() {
# search all regions
for region in $(get_regions); do
download_region "$region"
done
echo "Completed Downloading all the Lamdba Functions!"
}
case "${1:-}" in
help|-h|--help)
show_help
;;
download)
shift
get_function "$1" "$2"
;;
*)
shift
main
;;
esac
The original script worked for me on a Mac M1, ...BUT it deleted all the Lambda functions on AWS after downloading them.
It crashed my 64GB RAM mac, Twice :/