Todd Hayton thayton

thayton / keybase.md

Created September 11, 2017 14:16

I hereby claim:

To claim this, I am signing this object:

thayton / gist:76ee55882eaf26c8a0de6a1a01ef023e

Created March 9, 2018 14:18

	#!/usr/bin/env python

	import time
	import json
	import requests

	class Scraper(object):
	def __init__(self):
	self.url = 'http://preowned.ferrari.com/vdata'
	self.session = requests.Session()

thayton / rid_scraper.js

Last active October 9, 2020 17:51

Scraper for ASPX search form at https://myaccount.rid.org/Public/Search/Member.aspx

	/*
	* Companion code for article at http://toddhayton.com/2018/08/01/scraping-with-puppeteer/
	*
	* Setup:
	* $ mkdir scraper/
	* $ cd scraper/
	* $ npm init -y
	* $ npm install puppeteer --save
	*
	* Usage:

thayton / .gitignore

Last active February 12, 2019 16:38

Puppeteer scraper for https://l3com.taleo.net/careersection/l3_ext_us/jobsearch.ftl

node_modules/

thayton / l3com_scraper.py

Last active September 7, 2018 19:16

Python requests scraper for https://l3com.taleo.net/careersection/l3_ext_us/jobsearch.ftl#

	#!/usr/bin/env python

	import json
	import requests

	from bs4 import BeautifulSoup

	class L3ComScraper(object):
	def __init__(self):
	self.session = requests.Session()

thayton / campaignfinancemd.js

Last active September 14, 2018 20:40

Puppeteer scraper for https://campaignfinancemd.us/Public/ViewReceipts?theme=vista

	/*
	* Companion code for article at http://toddhayton.com/...
	*
	* Setup:
	* $ mkdir scraper/
	* $ cd scraper/
	* $ npm init -y
	* $ npm install puppeteer --save
	* $ npm install node-fetch --save
	* $ npm install progress-stream --save

thayton / l3com_scraper.js

Created February 12, 2019 16:46

	/*
	* Companion code for article at http://toddhayton.com/2019/02/12/revisiting-taleo-with-puppeteer/
	*
	* Setup:
	* $ mkdir scraper/
	* $ cd scraper/

thayton / lennar.py

Created February 28, 2019 15:24

	import json
	import requests

	class LennarScraper(object):
	def __init__(self):
	self.url = 'https://www.lennar.com/Services/Rest/SearchMethods.svc/GetInventoryTabDetails'
	self.session = requests.Session()
	self.data = {
	"CommunityID":"4531",
	"pageState":{

thayton / sheahomes.py

Last active February 28, 2019 19:03

	import re
	import json
	import urlparse
	import requests

	from bs4 import BeautifulSoup

	class SheaHomesScraper(object):
	def __init__(self):
	self.url = 'https://www.sheahomes.com/new-homes/colorado/denver-area/parker/stonewalk-at-stepping-stone/'

thayton / sheahomes.js

Created March 1, 2019 14:20

	/*
	* $ npm init -y
	* $ npm install puppeteer --save
	* $ node sheahomes.js
	*/
	const puppeteer = require('puppeteer');
	const url = 'https://www.sheahomes.com/new-homes/colorado/denver-area/parker/stonewalk-at-stepping-stone/';

	async function getLot(page, url) {
	await page.goto(url);