Skip to content

Instantly share code, notes, and snippets.

@sergiolucero
Last active March 7, 2022 13:00
Show Gist options
  • Select an option

  • Save sergiolucero/299b56ab912c8a4470a9bfb63d520085 to your computer and use it in GitHub Desktop.

Select an option

Save sergiolucero/299b56ab912c8a4470a9bfb63d520085 to your computer and use it in GitHub Desktop.
Mercurio de Valpo
import datetime, requests
import io, glob, os
import PyPDF2
from bs4 import BeautifulSoup
fecha = datetime.datetime.now().strftime('%Y%m%d')
print('FECHA:', fecha)
root='https://www.mercuriovalpo.cl/impresa/%s/%s/%s/' %(fecha[:4], fecha[4:6], fecha[6:])
base = root + 'papel'
bs = BeautifulSoup(requests.get(base).text, 'lxml')
links = set([link['href'] for link in bs.find_all('a') if root in link.get('href','')])
pdfWriter = PyPDF2.PdfFileWriter()
for link in sorted(links):
print(link)
bs2 = BeautifulSoup(requests.get(link).text, 'lxml')
pdf = bs2.find('section', attrs={'ref':'page'})['data-page']
pdfFileObj = io.BytesIO(requests.get(pdf).content)
pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
pdfWriter.addPage(pdfReader.getPage(0))
pdfWriter.write(open(f'MercurioValpo_{fecha}.pdf', 'wb'))
@sergiolucero

Copy link
Copy Markdown
Author

@sergiolucero

Copy link
Copy Markdown
Author

data.quant.cl:8080/valpo

soon as a chalice lambda!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment