Skip to content

Instantly share code, notes, and snippets.

@jayers99
Created March 18, 2019 18:46
Show Gist options
  • Save jayers99/203981c3690c44f013a51a6ded031bdc to your computer and use it in GitHub Desktop.
Save jayers99/203981c3690c44f013a51a6ded031bdc to your computer and use it in GitHub Desktop.
Tail aws vpc flow logs
#!/bin/bash
export AWS_PROFILE=default
FULLPATH="s3://jayers99-flowlog-test/AWSLogs/441655563487/vpcflowlogs/"
BUCKET=$(echo "$FULLPATH" | cut -d "/" -f3)
echo "#$BUCKET#";
echo ;
LASTFILES=$(aws s3 ls $FULLPATH --recursive | sort | tail -n 10 | awk '{print $4}')
rm ./lastfile
for i in $LASTFILES; do
if [ ! -f ./lastfile$i ]; then
aws s3 cp s3://$BUCKET/$i ./lastfile$i
fi
gunzip -c ./lastfile$i >> ./lastfile
done
cat ./lastfile | ggrep -P '^((?!version|NODATA).)*$' | awk '{ cmd="date -r "$11" +%Y-%m-%dT%H:%M:%S"; cmd | getline myTime; close(cmd); printf "%s %-15s %-15s\n", myTime, $4, $5 }' | sort
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment