Skip to content

Instantly share code, notes, and snippets.

@jineshpaloor
Created September 7, 2013 18:31
Show Gist options
  • Save jineshpaloor/6478011 to your computer and use it in GitHub Desktop.
Save jineshpaloor/6478011 to your computer and use it in GitHub Desktop.
python program to read a url and extract its meta keyword and meta description
from bs4 import BeautifulSoup
import requests
def main():
#r = requests.get('http://www.aurionpro.com/')
r = requests.get('http://www.sourcebits.com/')
soup = BeautifulSoup(r.content, "html")
title = soup.title.string
print 'TITLE IS :', title
meta = soup.find_all('meta')
for tag in meta:
if 'name' in tag.attrs.keys() and tag.attrs['name'].strip().lower() in ['description', 'keywords']:
print 'NAME :',tag.attrs['name'].lower()
print 'CONTENT :',tag.attrs['content']
if __name__ == '__main__':
main()
@erfelipe
Copy link

Thanks for the code.

@mikesingh7987
Copy link

Thanks for the code, how do we add a for loop to the code if we have multiple URLs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment