Skip to content

Instantly share code, notes, and snippets.

@mtholder
Created November 28, 2014 12:48
Show Gist options
  • Select an option

  • Save mtholder/e0585d5d756d21acaa2c to your computer and use it in GitHub Desktop.

Select an option

Save mtholder/e0585d5d756d21acaa2c to your computer and use it in GitHub Desktop.
chars_in_file.py takes a list of filepaths (to files encoded using utf-8). prints a list of the characters encountered.
#!/usr/bin/env python
import codecs
import sys
chars = set()
for filepath in sys.argv[1:]:
with codecs.open(filepath, 'r', encoding='utf-8') as fo:
for line in fo:
chars.update(iter(line))
c = list(chars)
c.sort()
for i in c:
sys.stdout.write(u'{}\t {}\n'.format(i, repr(i)).encode('utf-8'))
sys.stderr.write('{} different characters found\n'.format(len(c)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment