Skip to content

Instantly share code, notes, and snippets.

@narate
Created September 9, 2015 11:06
Show Gist options
  • Save narate/d285f9ea5bafcb3dc831 to your computer and use it in GitHub Desktop.
Save narate/d285f9ea5bafcb3dc831 to your computer and use it in GitHub Desktop.
Get all html meta tag with facebookexternalhit/1.1 user agent
#!/usr/bin/env python
from BeautifulSoup import BeautifulSoup
import urllib2
import sys
if len(sys.argv) < 2 :
print 'Usage : fbbot.py URL'
exit()
http = urllib2.build_opener()
http.addheaders = [('User-agent', 'facebookexternalhit/1.1')]
try :
page = http.open(sys.argv[1])
soup = BeautifulSoup(page.read())
for m in soup.findAll('meta'):
print m
except ValueError:
print 'Invalid URL'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment