Skip to content

Instantly share code, notes, and snippets.

@JerryFox
Created March 8, 2015 23:37
Show Gist options
  • Select an option

  • Save JerryFox/74f8749d2c8ebe1da48a to your computer and use it in GitHub Desktop.

Select an option

Save JerryFox/74f8749d2c8ebe1da48a to your computer and use it in GitHub Desktop.
bash - finding duplicities using md5sum (for NTI / OSU-P, Ing. Jana Kolaja Ehlerová, Ph.D.)
#!/bin/bash
# the same contents file list
# finding using md5sum
#echo $1
if [ "$1" = "--help" ]; then
echo "here will be some help..."
echo "using: "
echo "mydiff [directory]"
else
if [ "$1" = "" ]; then
ADR="."
else
# echo "explicit input processing..."
ADR=$1
fi
#echo $ADR
#ls $ADR
# temporary directory creating
TEMP=$( mktemp -d )
# insert file name into list according to its hash
# for each hash one file
find $ADR -type f |
while read f; do
HASH=$(md5sum $f | cut -f1 -d' ' )
NAME=$(md5sum $f | cut -f3 -d' ' )
echo $NAME >> $TEMP/$HASH
done
# file lists processing
for i in $(find $TEMP -type f); do
DUPL=$(cat $i | wc -l)
if (( $DUPL > 1 )); then
echo -n "hash: "$(echo $i | cut -d"/" -f4)" - "
echo $DUPL" files with equivalent content"
cat $i
fi
done
# temporary directory removing
rm -rf $TEMP
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment