Skip to content

Instantly share code, notes, and snippets.

@karlcow
Created March 6, 2013 04:03
Show Gist options
  • Select an option

  • Save karlcow/5096609 to your computer and use it in GitHub Desktop.

Select an option

Save karlcow/5096609 to your computer and use it in GitHub Desktop.
Issue with user agent sniffing, python and wikipedia
→ python3.3

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 01:25:11) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>> opener = urllib.request.build_opener()
>>> opener.addheaders = [('User-agent', 'Python-urllib/3.3')]
>>> fobj = opener.open('http://en.wikipedia.org/robots.txt')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 479, in open
  response = meth(req, response)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 591, in http_response
  'http', request, response, code, msg, hdrs)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 517, in error
  return self._call_chain(*args)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 451, in _call_chain
  result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/urllib/request.py", line 599, in http_error_default
  raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
>>> print(fobj.getcode())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'fobj' is not defined

Identified in issue 15851 in Python project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment