Skip to content

Instantly share code, notes, and snippets.

@jzellman
Created August 26, 2009 14:34
Show Gist options
  • Save jzellman/175534 to your computer and use it in GitHub Desktop.
Save jzellman/175534 to your computer and use it in GitHub Desktop.
#! /usr/bin/env python
# Reads in a file from stdin and converts its encoding from 8859 to utf-8
# Writes to stdout and lines that fail to stderr
# Example usage ./convert.py < in.txt 1> out.txt 2> errors.txt
import sys
for line_number, line in enumerate(sys.stdin.readlines()):
try:
sys.stdout.write(unicode(line, "8859").encode("utf-8"))
except UnicodeDecodeError as e:
sys.stderr.write("%5d %s"%(line_number + 1, line))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment