Skip to content

Instantly share code, notes, and snippets.

@nicc777
Created April 10, 2020 04:29
Show Gist options
  • Save nicc777/cb81783ab50a7fdf5c1882b6bf9b3523 to your computer and use it in GitHub Desktop.
Save nicc777/cb81783ab50a7fdf5c1882b6bf9b3523 to your computer and use it in GitHub Desktop.
Solution for getting all S3 buckets and their size

So, the concept is to get a summary of the size of each S3 bucket - almost like the du command on Linux.

The code concept from below is still untested, but I will soon test it and hopefulle have a modified general purpose utility for wider use.

Basic code from stackexchange.com:

#!/bin/bash
aws_profile=('profile1' 'profile2' 'profile3');

#loop AWS profiles
for i in "${aws_profile[@]}"; do
  echo "${i}"
  buckets=($(aws --profile "${i}" --region your_region s3 ls s3:// --recursive | awk '{print $3}'))

  #loop S3 buckets
  for j in "${buckets[@]}"; do
  echo "${j}"
  aws --profile "${i}" --region your_region s3 ls s3://"${j}" --recursive --human-readable --summarize | awk END'{print}'
  done

done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment