aws-cli get total size of all objects within s3 prefix. (mimic behavior of `s3cmd du` with aws-cli)

aws-cli-s3cmd-du.sh

#!/bin/bash

function s3du(){
   bucket=`cut -d/ -f3 <<< $1`
   prefix=`awk -F/ '{for (i=4; i<NF; i++) printf $i"/"; print $NF}' <<< $1`
   aws s3api list-objects --bucket $bucket --prefix=$prefix --output json --query '[sum(Contents[].Size), length(Contents[])]' | jq '. |{ size:.[0],num_objects: .[1]}'
}

s3du $1;

If you want to sum by top level prefixes within a bucket, you can also try something like this

i tried running the script as is, but i couldn't succeed, can you help me with this. Thanks

My not-so-terse adaption for printing human-readable sizes:

function aws3du(){
  bucket=`cut -d/ -f3 <<< $1`
  prefix=`awk -F/ '{for (i=4; i<NF; i++) printf $i"/"; print $NF}' <<< $1`
  aws s3api \
    list-objects \
    --bucket $bucket \
    --prefix=$prefix \
    --output text \
    --query '[sum(Contents[].Size), length(Contents[])]' \
    | while read -r size num_objects; do
      jq '. |{ size:.[0],num_objects: .[1]}' <<< "[\"$(numfmt --to=si ${size})\",${num_objects}]"
     done
}

Usage:

❯ aws3du ${s3path}
{
  "size": "328K",
  "num_objects": 1
}

Will be made simpler if something like this gets implemented: jqlang/jq#147