Skip to content

Instantly share code, notes, and snippets.

@Lukasa
Created March 31, 2013 16:28
Show Gist options
  • Select an option

  • Save Lukasa/5281176 to your computer and use it in GitHub Desktop.

Select an option

Save Lukasa/5281176 to your computer and use it in GitHub Desktop.
Requests and Beautiful Soup example, following the form of http://bpaste.net/show/kMetvCdrfnzh5RgiUKU4/
from BeautifulSoup import BeautifulSoup
import requests
import urlparse
URL = 'example.com'
s = requests.Session()
def fetch(url, data=None):
if data is None:
return s.get(url).content
else:
return s.post(url, data=data).content
soup = BeautifulSoup(fetch(URL))
form = soup.find('form')
fields = form.findAll('input')
formdata = dict( (field.get('name'), field.get('value')) for field in fields)
formdata['username'] = u'username'
formdata['password'] = u'password'
print formdata
posturl = urlparse.urljoin(URL, form['action'])
print posturl
r = s.post(posturl, data=formdata)
print r.text
print s.get(URL).text
@IMMACORP
Copy link
Copy Markdown

IMMACORP commented Jul 4, 2020

Thanks! This is helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment