Skip to content

Instantly share code, notes, and snippets.

@xman1980
Created May 3, 2017 08:00
Show Gist options
  • Save xman1980/563b46cebf655db884d59646934329f9 to your computer and use it in GitHub Desktop.
Save xman1980/563b46cebf655db884d59646934329f9 to your computer and use it in GitHub Desktop.
Finding directories older than N days in HDFS
#!/bin/bash
usage="Usage: dir_diff.sh [days]"
if [ ! "$1" ]
then
echo $usage
exit 1
fi
now=$(date +%s)
curl "http://localhost:50070/getimage?getimage=1&txid=latest" > img.dump
hdfs oiv -i img.dump -o fsimage.txt
cat fsimage.txt | grep "^d" | while read f; do
dir_date=`echo $f | awk '{print $6}'`
difference=$(( ( $now - $(date -d "$dir_date" +%s) ) / (24 * 60 * 60 ) ))
if [ $difference -gt $1 ]; then
echo $f;
fi
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment