Last active
January 31, 2024 12:04
-
-
Save Gershon-A/5fadb55289c0aa6f265d155bf46a6758 to your computer and use it in GitHub Desktop.
This script lists all objects in a specified AWS S3 bucket that were moved to the GLACIER storage class within a specified number of days ago.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# This script lists all objects in a specified AWS S3 bucket that were moved to the GLACIER storage class within a specified number of days ago. | |
# Usage: ./show_obgects_moved_to_glacier.sh your-bucket-name aws-profile days_ago | |
# Get the arguments | |
bucket="$1" | |
profile="$2" | |
days_ago="$3" | |
# Set the page size for pagination | |
page_size=100 | |
# Initialize a counter for the number of objects | |
count=0 | |
# Initialize a counter for the number of objects processed | |
processed=0 | |
# Calculate the date days_ago days ago in Unix timestamp in UTC | |
date_days_ago=$(date -u -d "$days_ago days ago" +%s) | |
# List all objects in your bucket | |
while read -r object; do | |
key=$(echo "$object" | jq -r '.[0]') | |
last_modified=$(echo "$object" | jq -r '.[1]') | |
storage_class="GLACIER" | |
# Convert the LastModified date to a Unix timestamp | |
last_modified_unix=$(date -d"$last_modified" -u +%s) | |
# Debug: Print the key, last modified date, and storage class of the object for debugging | |
# echo "Key: $key, Last Modified: $last_modified, Storage Class: $storage_class" | |
# Check if the storage class is GLACIER and the LastModified date is greater than or equal to date_days_ago | |
if [ "$last_modified_unix" -ge "$date_days_ago" ]; then | |
# Print the key of the object | |
echo "Key: $key" | |
# Increment the counter | |
((count++)) | |
fi | |
# Increment the processed counter | |
((processed++)) | |
# Print the progress every 100 objects | |
if (( processed % 100 == 0 )); then | |
echo "Processed $processed objects so far" | |
fi | |
done < <(aws s3api list-objects-v2 --bucket $bucket --query 'Contents[?StorageClass==`GLACIER`].[Key, LastModified]' --output json --profile $profile --page-size $page_size | jq -c '.[]') | |
# Print the summary | |
echo "Total number of objects moved to GLACIER in the last $days_ago days: $count" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Script: show_objects_moved_to_glacier.sh
This script lists all objects in a specified AWS S3 bucket that were moved to the GLACIER storage class within a specified number of days ago.
Usage
Parameters
<your-bucket-name>
: The name of the AWS S3 bucket to check.<aws-profile>
: The AWS profile to use for AWS CLI commands.<days_ago>
: The number of days ago to check for objects moved to GLACIER.How it works
<days_ago>
days ago in Unix timestamp in UTC.LastModified
date.LastModified
date is converted to a Unix timestamp.LastModified
date is greater than or equal to the date<days_ago>
days ago. If it is, the script increments a counter.