ANSI / UTF-8 (with or without BOM) conversion #Windows

ansi-utf8-conversion.md

Using Uni2Me

• It's free but discontinued.

Using UTFCast

• Proprietary software
• Allows conversion from ANSI to UTF-8 with or without BOM

Using Notepad++

Using Python Script Plugin

from glob import glob
from Npp import notepad

globPath = "C:\MyFiles\*.txt"

for file in glob(globPath):
  notepad.open(file)
  notepad.runMenuCommand("Encoding", "Convert to UTF-8-BOM")
  notepad.save()
  notepad.close()

Using Macros

1. Start Macro recording
2. Select Encoding > Convert to UTF-8-BOM
3. Select all text and copy it (it's a bug otherwise it will replace file contents with Clipboard content)
4. Save file and close it

Using Bash

Add BOM to an already encoded UTF-8 file

echo -ne '\xEF\xBB\xBF' > utf8-no-bom.txt

Batch conversion using find and iconv

# Find all .txt files and convert them to UTF-8 (assuming US characters only / ANSI)
find *.txt -exec 'iconv -f CP1252 -t UTF-8  {} > {}'

# all Windows character sets
iconv -l | grep -i windows

Batch conversion using ls and iconv

for i in `ls *.txt`; do
  iconv -f WINDOWS-1252 -t UTF8 $i -o $i.utf8
  mv $i.utf8 $i
done

Change in …; do with in $@; do to create a usable Bash file. (e.g. convert.sh myfile.txt myfile2.txt)

Using Batch

Add BOM to all text files using nkf

for %a in (*.txt) do nkf32 -W8 -w8 --overwrite "%a"

Download binary for Windows • Source code

Note: Change -w8 with -w80 to remove BOM

Batch conversion using for and iconv

for %a in (*.txt) do iconv -f CP1252 -t UTF-8 "%a" > "%a"

Trivial methods

• Using paid software Replace Pioneer

Thanks for sharing the method to change the file encoding using Python in NP++ 👍

I had to make some adjustments to the script to get it to work.
First of all I had to change the globPath to be set like this:

globPath = "C:\\mydir\\*.txt"

And the line notepad.runMenuCommand needs to be adjusted to the NP++ language in use.
So I had to change it on my German NP++ setup from:

notepad.runMenuCommand("Encoding", "Convert to UTF-8")

to

notepad.runMenuCommand("Kodierung", "Konvertiere zu UTF-8")

Now there is a much simpler, free and open source tool, thanks to @tomwillow :)
SmartCharsetConverter