Skip to content

Instantly share code, notes, and snippets.

@juji

juji/cleanUTF8.js

Last active April 21, 2017 07:43
Show Gist options
  • Save juji/9ebf97f534e1fbaa26eeed48343d9457 to your computer and use it in GitHub Desktop.
Save juji/9ebf97f534e1fbaa26eeed48343d9457 to your computer and use it in GitHub Desktop.
Download ZIP
clean non utf8 character
Raw
cleanUTF8.js
function cleanUTF8(str){
return str.replace(/[\x00-\x08\x10\x0B\x0C\x0E-\x19\x7F]|[\x00-\x7F][\x80-\xBF]+|([\xC0\xC1]|[\xF0-\xFF])[\x80-\xBF]*|[\xC2-\xDF]((?![\x80-\xBF])|[\x80-\xBF]{2,})|[\xE0-\xEF](([\x80-\xBF](?![\x80-\xBF]))|(?![\x80-\xBF]{2})|[\x80-\xBF]{3,})/g,'')
.replace(/\xE0[\x80-\x9F][\x80-\xBF]|\xED[\xA0-\xBF][\x80-\xBF]/g,'');
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment