yyyyyyuanfei/rm_duplicates.sh

Created January 8, 2012 07:26

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/yyyyyyuanfei/1577583.js"></script>
Save yyyyyyuanfei/1577583 to your computer and use it in GitHub Desktop.

Download ZIP

remove duplications in osx

Raw

rm_duplicates.sh

OUTF=rem-duplicates.sh; echo "#! /bin/sh" > $OUTF; find . "$@" -type f -print0 | xargs -0 -n1 gmd5sum | sort --key=1,32 | guniq -w 32 -d --all-repeated=separate | gsed -r 's/^[0-9a-f]*( )*//;s/([^a-zA-Z0-9./_-])/\\\1/g;s/(.+)/#rm \1/' >> $OUTF; chmod a+x $OUTF; ls -l $OUTF

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment