Skip to content

Instantly share code, notes, and snippets.

@worenga
Created July 17, 2014 11:35
Show Gist options
  • Save worenga/44b5df923a622424eeb7 to your computer and use it in GitHub Desktop.
Save worenga/44b5df923a622424eeb7 to your computer and use it in GitHub Desktop.
encoding fixer
#!/bin/bash
#small encoding fixer by benedikt wolters<[email protected]>
#either write iconv output conversion to dev null or pipe it to conversion
#Debug mode?
#set -x
echo Will fix encoding for text files... this may take a while depending on mess level
BAD_FILES=$(find . \
-type f \
-print| \
xargs file|grep text|cut -d: -f1 |xargs -I {} bash -c "iconv -f utf-8 -t utf-16 {} &>/dev/null || echo {}"|\
xargs -I {} bash -c "file -bi {} |grep iso-8859-1 &>/dev/null && echo {}"
#xargs -I {} bash -c "iconv -f iso-8859-1 -t utf-8 -c {} > {}"
)
for file in $BAD_FILES
do
echo "Will fix... $file"
iconv -f iso-8859-1 -t utf8 -o "$file.new" "$file" && mv -f "$file.new" "$file"
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment