Skip to content

Instantly share code, notes, and snippets.

@HQMIS
Last active December 10, 2015 08:38
Show Gist options
  • Select an option

  • Save HQMIS/4408574 to your computer and use it in GitHub Desktop.

Select an option

Save HQMIS/4408574 to your computer and use it in GitHub Desktop.
【Python】【Library】【chardet】
1、模块作用
  chardet模块用来实现字符串/文件编码检测
2、chardet下载与安装
  下载地址:http://pypi.python.org/pypi/chardet/
  下载chardet后,解压chardet压缩包,直接将chardet文件夹放在应用程序目录下,就可以使用import chardet开始使用chardet了。
  也可以将chardet拷贝到Python系统目录下,这样你所有的python程序只要用import chardet就可以了。
  Python第三方模块中一般会自带setup.py文件,在CMD里切换目录至chardet,然后执行:python setup.py install。
  如果执行上述命令,报错:ImportError: No module named setuptools,则要进行setuptools的安装。
3、setuptools下载与安装
  下载地址:http://pypi.python.org/pypi/setuptools#downloads
  下载setuptools-0.6c11.win32-py2.7.exe,直接双击exe文件,进行安装。
  安装完成之后,即可直接使用python setup.py install安装Python Library啦。
4、使用实例
  chardet.detect()返回字典,其中confidence是检测精确度,encoding是编码形式
  (1)网页编码判断:
Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import chardet
>>> import urllib
>>> rawdata = urllib.urlopen('http://www.byhh.net/').read()
>>> chardet.detect(rawdata)
{'confidence': 0.99, 'encoding': 'GB2312'}
  (2)文件编码判断
Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import chardet
>>> f = open("C://log.txt", "r")
>>> r = f.readline()
>>> print chardet.detect(r)
{'confidence': 1.0, 'encoding': 'ascii'}
(3)变量编码判断
Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import chardet
>>> strPowerManage = "电源管理"
>>> chardet.detect(strPowerManage)
{'confidence': 0.99, 'encoding': 'GB2312'}
@HQMIS

HQMIS commented Dec 29, 2012

Copy link
Copy Markdown
Author

Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.

import chardet
import requests
r = requests.get('http://www.byhh.net')
chardet.detect(r.text)

Warning (from warnings module):
File "E:\Python27\lib\site-packages\chardet\universaldetector.py", line 90
if aBuf[:len(chunk)] == chunk:
UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
{'confidence': 1.0, 'encoding': 'ascii'}

r.encoding
r.headers['content-type']
'text/html'

@HQMIS

HQMIS commented Dec 29, 2012

Copy link
Copy Markdown
Author

requests中, r.text返回的值, 都是转码过的?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment