Skip to content

Instantly share code, notes, and snippets.

@peteristhegreat

peteristhegreat/clean_files.sh

Created August 13, 2019 20:31
Show Gist options
  • Save peteristhegreat/8191355594db5bd466489849965adf4c to your computer and use it in GitHub Desktop.
Save peteristhegreat/8191355594db5bd466489849965adf4c to your computer and use it in GitHub Desktop.
Download ZIP
Remove Unicode Characters from a list of files
Raw
clean_files.sh
#!/usr/bin/env bash
while read p; do
echo "$p"
#cat "$p" | gtr -d '\200-\377' | sponge "$p"
#iconv -f utf-8 -t ascii//TRANSLIT "$p" | sponge "$p"
vim +"set nobomb | set fenc=utf8 | x" "$p"
done <files.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment