Last active
October 18, 2023 16:47
-
-
Save opera443399/0fb9b7b4ce3f37045fdd33639279e6a1 to your computer and use it in GitHub Desktop.
export PYTHONIOENCODING=UTF-8
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from: | |
http://chase-seibert.github.io/blog/2014/01/12/python-unicode-console-output.html | |
Print to the console in Python without UnicodeEncodeErrors 12 Jan 2014 | |
I can't believe I just found out about this! If you use Python with unicode data, such as Django database records, you may have seen cases where you print a value to the console, and if you hit a record with an extended (non-ascii) character, your program crashes with the following: | |
Traceback (most recent call last): | |
File "foobar.py", line 792, in <module> | |
print value | |
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 20: ordinal not in range(128) | |
This often comes up when you have a Django model instance with a user-entered field in the __unicode__ return value. In the past, I have solved this by changing the print statement to print something that cannot be unicode, such as a database ID. Alternatively, you could do something like value.encode('ascii', 'ignore'). But just this week, I realized that there is a much better solution. | |
Your termianl can typically handle UTF8 characters just fine. The issue is actually that Python is just getting confused about what encoding the terminal accepts. However, you can explicitly set this value. | |
The PYTHONIOENCODING environment variable controls what encoding stdout/stderr uses. If you do an export PYTHONIOENCODING=UTF-8, it will solve the problem. You can also prefix any given single python command-line invocation with this value, such as PYTHONIOENCODING=UTF-8 ./manage.py runserver. | |
Alternatively, you can set this value in your actual code, say in your settings.py file: | |
import sys | |
import codecs | |
sys.stdout = codecs.getwriter('utf8')(sys.stdout) | |
sys.stderr = codecs.getwriter('utf8')(sys.stderr) | |
I'm currently working at NerdWallet, a startup in San Francisco trying to bring clarity to all of life's financial decisions. We're hiring like crazy. Hit me up on Twitter, I would love to talk. | |
Follow @chase_seibert on Twitter |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Thanks, This helped me 👍