Skip to content

Instantly share code, notes, and snippets.

@spyros12
spyros12 / crawler.py
Created August 30, 2020 15:19 — forked from AO8/crawler.py
Crawl a website and gather all internal links with Python and BeautifulSoup.
# Adapted from example in Ch.3 of "Web Scraping With Python, Second Edition" by Ryan Mitchell
import re
import requests
from bs4 import BeautifulSoup
pages = set()
def get_links(page_url):
global pages
@AO8
AO8 / crawler.py
Last active May 23, 2023 09:12
Crawl a website and gather all internal links with Python and BeautifulSoup.
# Adapted from example in Ch.3 of "Web Scraping With Python, Second Edition" by Ryan Mitchell
import re
import requests
from bs4 import BeautifulSoup
pages = set()
def get_links(page_url):
global pages
@filipkral
filipkral / python-cheat-sheet-basic.py
Last active November 14, 2024 05:32
Basic Python Cheat Sheet
#!/usr/bin/env python
"""Basic Python Cheat Sheet by Filip Kral on 2015/02/16"""
"""
Python is a cross-platform, interpreted, object-oriented programming language.
That means you can run it on Linux, Windows, Mac, and other platforms,
you don't need to compile your code to execute it because it is compiled on
the fly, and you can use classes and objects.