Skip to content

Instantly share code, notes, and snippets.

View starhound's full-sized avatar
🎆

Wesley Reid starhound

🎆
View GitHub Profile
@starhound
starhound / scraper.py
Last active September 20, 2024 14:31
OASIS Doc Scraper
import os
import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin, urlparse
import time
BASE_URL = 'https://docs.oasis-open.org/'
VISITED = []