Skip to content

Instantly share code, notes, and snippets.

@curiousleo
Last active August 29, 2015 14:27
Show Gist options
  • Select an option

  • Save curiousleo/faf6bff70e6faed041cb to your computer and use it in GitHub Desktop.

Select an option

Save curiousleo/faf6bff70e6faed041cb to your computer and use it in GitHub Desktop.
#!/usr/bin/env python
from bs4 import BeautifulSoup
from pocket import Pocket
import requests
from urllib.parse import urljoin
consumer_key = '44711-81de59a4a614b97a4b76dc9a'
access_token = 'XXX' # put your access token here
starturl = 'https://www.economist.com/printedition'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0'}
req = requests.get(starturl, headers=headers)
soup = BeautifulSoup(r.content, 'html.parser')
urls = [ urljoin(req.url, link['href']) for link in soup.select('a.node-link') ]
p = Pocket(consumer_key, access_token)
for url in urls:
p.add(url)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment