Skip to content

Instantly share code, notes, and snippets.

View StanGirard's full-sized avatar
🧠
Building

Stan Girard StanGirard

🧠
Building
View GitHub Profile
@StanGirard
StanGirard / sitemap-crawler.py
Created March 2, 2020 21:33
Sitemap Crawler Python
###################
# You can find the article about this gist here:
# https://primates.dev/find-all-urls-of-a-website-in-a-few-seconds-python/
####################
import requests
from bs4 import BeautifulSoup as Soup
import pandas as pd
import hashlib
@StanGirard
StanGirard / Parsing an API XML Response Data
Created February 27, 2020 22:06
Parsing an API XML Response Data - Python
import requests
import xml.etree.ElementTree as ET
r = requests.get('https://www.washingtonpost.com/arcio/news-sitemap/')
print(r.content)
xmlDict = {}
root = ET.fromstring(r.content)
for child in root.iter('*'):
print(child.tag)
for sitemap in root:
children = sitemap.getchildren()