python3 and scrapy (pip install scrapy)
scrapy runspider -o items.csv -a site="https://yoursite.org" 1spider.pypython3 2format_results.py
Mike McDougal mcdoogs
mcdoogs
/ 0readme.md
Last active
April 30, 2019 15:24
— forked from mdamien/0readme.md
404 link detector with scrapy