Skip to content

Instantly share code, notes, and snippets.

@kennethreitz
Created May 16, 2011 00:17
Show Gist options
  • Save kennethreitz/973705 to your computer and use it in GitHub Desktop.
Save kennethreitz/973705 to your computer and use it in GitHub Desktop.
urllib2 vs requests
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import urllib2
gh_url = 'https://api.github.com'
req = urllib2.Request(gh_url)
password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_manager.add_password(None, gh_url, 'user', 'pass')
auth_manager = urllib2.HTTPBasicAuthHandler(password_manager)
opener = urllib2.build_opener(auth_manager)
urllib2.install_opener(opener)
handler = urllib2.urlopen(req)
print handler.getcode()
print handler.headers.getheader('content-type')
# ------
# 200
# 'application/json'
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import requests
r = requests.get('https://api.github.com', auth=('user', 'pass'))
print r.status_code
print r.headers['content-type']
# ------
# 200
# 'application/json'
@itczl22
Copy link

itczl22 commented Nov 10, 2021

nice

@abdallaabdalrhman
Copy link

import requests
response = requests.get("https://en.wikipedia.org/wiki/Python")
print(response)
print(response.status_code)

@cyrusDev1
Copy link

Really simple

@HGStyle
Copy link

HGStyle commented Dec 31, 2022

URLLIB (Python3)

from urllib import urlopen
r = urlopen('https://google.com/')
print(r.read().decode())
print(r.code)

Requests (Python3)

from requests import get
r = get('https://google.com/')
print(r.text)
print(r.status_code)

4 lines in each code !
But yes, when (like in the example) you want to login to a website, thats annother thing...
URLLIB is great for little things if you cant download Requests.
Requests is really better because Python code with Requests its a lot more human-readable.

@Farzin300
Copy link

0_urllib2.py
#!/usr/bin/env python

-- coding: utf-8 --

import urllib2

gh_url = 'https://api.github.com'

req = urllib2.Request(gh_url)

password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_manager.add_password(None, gh_url, 'user', 'pass')

auth_manager = urllib2.HTTPBasicAuthHandler(password_manager)
opener = urllib2.build_opener(auth_manager)

urllib2.install_opener(opener)

handler = urllib2.urlopen(req)

print handler.getcode()
print handler.headers.getheader('content-type')

------

200

'application/json'``

[[](https://perfectbt. com/crash/php)

@lsloan
Copy link

lsloan commented Mar 14, 2024

@ HGStyle wrote…

4 lines in each code ! But yes, when (like in the example) you want to login to a website, thats annother thing... URLLIB is great for little things if you cant download Requests. Requests is really better because Python code with Requests its a lot more human-readable.

Yes, that's the point of @kennethreitz's examples. Operations that are complicated with urllib are simple with requests.

@HGStyle
Copy link

HGStyle commented Mar 14, 2024

@source-creator wrote...

Unfortunately, code to format an url with requests is longer and more bloated than urllib:

# requests
from requests import Request
url = Request(None, 'http://example.com/?', params={'Data1': 'data'}).prepare().url

# urllib
import urllib
url = 'http://example.com/?' + urllib.parse.urlencode({'Data1': 'data'})

Just read this: https://stackoverflow.com/a/46783596
You can just use requests.utils (and other submodules, I've seen annother one for functions included in urllib.parse but I don't remember it) instead of urllib.parse in most cases I think.
(and it does not requires requests.utils to be imported because importing requests also imports requests.utils)
NOTE: The functions are comming right from the URLLIB package, so the actual documentation is in URLLIB for some (not all) of the function of this submodule of requests.

@LIghtJUNction
Copy link

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment