Skip to content

Instantly share code, notes, and snippets.

@screeley

screeley/change_user_agents.py

Created May 31, 2011 20:02
Show Gist options
  • Save screeley/1001152 to your computer and use it in GitHub Desktop.
Save screeley/1001152 to your computer and use it in GitHub Desktop.
Download ZIP
Change.org is changing responses for Facebook and Google
Raw
change_user_agents.py
## Change.org. We are going to start using facebook's user-agent. FYI.
In [1]: import urllib2
In [2]: req = urllib2.Request('http://www.change.org/petitions/boston-red-sox-please-make-an-it-gets-better-video')
#Embedly's User Agent
In [3]: req.add_header('User-Agent', 'Mozilla/5.0 (compatible; Embedly/0.2; +http://support.embed.ly/)')
In [4]: urllib2.urlopen(req).read()
#Random HTML that sets cookies
Out[4]: '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html><head><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"><meta http-equiv="Content-Script-Type" content="text/javascript"><script type="text/javascript">function setCookie(c_name, value, expiredays) { var exdate = new Date(); exdate.setDate(exdate.getDate()+expiredays); document.cookie = c_name + "=" + escape(value) + ((expiredays==null) ? "" : ";expires=" + exdate.toGMTString()) + ";path=/"; } function getHostUri() { var loc = document.location; return loc.toString(); } setCookie(\'YPF8827340282Jdskjhfiw_928937459182JAX666\', \'204.9.220.41\', 10); location.href = getHostUri();</script></head><body><noscript>This site requires JavaScript and Cookies to be enabled. Please change your browser settings or upgrade your browser.</noscript></body></html>\n'
#Chrome User-Agent
In [5]: req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/3.0.195.38 Safari/532.5')
In [6]: urllib2.urlopen(req).read()
#Random HTML that sets cookies
Out[6]: '<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html><head><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"><meta http-equiv="Content-Script-Type" content="text/javascript"><script type="text/javascript">function setCookie(c_name, value, expiredays) { var exdate = new Date(); exdate.setDate(exdate.getDate()+expiredays); document.cookie = c_name + "=" + escape(value) + ((expiredays==null) ? "" : ";expires=" + exdate.toGMTString()) + ";path=/"; } function getHostUri() { var loc = document.location; return loc.toString(); } setCookie(\'YPF8827340282Jdskjhfiw_928937459182JAX666\', \'204.9.220.41\', 10); location.href = getHostUri();</script></head><body><noscript>This site requires JavaScript and Cookies to be enabled. Please change your browser settings or upgrade your browser.</noscript></body></html>\n'
#Facebook's User-Agent
In [7]: req.add_header('User-Agent', 'facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)')
In [8]: urllib2.urlopen(req).read()
#Full HTML in all it's glory
Out[8]: '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n<!--[if IE 7]> <html class="no-js ie7" lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml"> <![endif]-->\n<!--[if IE 8]> <html class="no-js ie8" lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml"> <![endif]-->\n<!--[if IE 9]> <html class="no-js ie9" lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml"> <![endif]-->\n<!--[if (gt IE 9)|!(IE)]><!--> <html class="no-js not-ie" lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml"> <!--<![endif]-->\n<head>\n<title>Gay Rights Petition: Boston Red Sox: Please make an It Gets Better video! | Change.org</title>\n<meta charset=\'utf-8\' />\n<meta content=\'IE=edge,chrome=1\' http-equiv=\'X-UA-Compatible\' />\n<meta content=\'width=device-width, initial-scale=1.0, maximum-scale=1.1\' name=\'viewport\' />\n<meta content=\'yes\' name=\'apple-mobile-web-app-capable\' />\n<meta content=\'black\' name=\'apple-mobile-web-app-status-bar-style\' />\n<meta conte.....
#Google Bot User-Agent
In [9]: req.add_header('User-Agent', 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)')
In [10]: urllib2.urlopen(req).read()
#Full HTML in all it's glory
Out[10]: '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n<!--[if IE 7]> <html class="no-js ie7" lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml"> <![endif]-->\n<!--[if IE 8]> <html class="no-js ie8" lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml"> <![endif]-->\n<!--[if IE 9]> <html class="no-js ie9" lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml"> <![endif]-->\n<!--[if (gt IE 9)|!(IE)]><!--> <html class="no-js not-ie" lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml"> <!--<![endif]-->\n<head>\n<title>Gay Rights Petition: Boston Red Sox: Please make an It Gets Better video! | Change.org</title>\n<meta charset=\'utf-8\' />\n<meta content=\'IE=edge,chrome=1\' http-equiv=\'X-UA-Compatible\' />\n<meta content=\'width=device-width, initial-scale=1.0, maximum-scale=1.1\' name=\'viewport\' />\n<meta content=\'yes\' name=\'apple-mobile-web-app-capable\' />\n<meta content=\'black\' name=\'apple-mobile-web-app-status-bar-style\' />\n<meta content="Every day, gay and lesbian teens in Middle School and High School are made fun of and bullied. It\'s sad that some of them are bullied so badly, they..." name=\'description\' />\n<meta content=\'petitions, online petitions\' name=\'keywords\' />\n<link href=\'http://change-production.s3.amazonaws.com/photos/5/mm/ie/DammiEFWyYyRcDY-111x83-cropped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment