Skip to content

Instantly share code, notes, and snippets.

View dwillis's full-sized avatar

Derek Willis dwillis

View GitHub Profile
require 'rubygems'
require 'fech'
filing = Fech::Filing.new(824973) # the Aflac PAC's pre-general filing
filing.download
filing.summary
@dwillis
dwillis / senate.py
Created January 12, 2013 16:01
A Python script to parse Senate votes, by Aaron Swartz, 2005. Thank you, and may you find peace.
#Module to retrieve and parse votes from Senate web site, output to stdout
import urllib, re
f=open('senate03.txt','w')
sr = re.compile('<question>(.*?)</td>.*?<b> Vote Date: </b><.*?>(.*?)</td>.*?<b> Required For Majority: </b><td class="contenttext">(.*?)</td></td><td valign="top" class="contenttext"><b> Vote Result: </b><.*?>(.*?)</td></td>', re.S)
sr2 = re.compile('</span>\n<TABLE width="100%"(.*?)</TABLE>', re.S)
def self.max_reports
where("report_num IN (SELECT max(report_num) from mytable)")
end
@dwillis
dwillis / state_assignments
Last active December 14, 2015 02:49
State assignments for Data Reporting Midterm
Name State
Gwen MI
Varsha MA
Michael VA
Natalia NC
Nayana AZ
Terry IN
@dwillis
dwillis / presdocs-lakers
Created February 27, 2013 20:52
Lakers
@lakers = Document.category("Lakers")
=> [#<Presdocs::Document:0x007ff83a24c1f8 @id="DCPD-201001065", @city="Washington", @state="DC", @lat=nil, @lng=nil, @title="Remarks Honoring the 2010 National Basketball Association Champion Los Angeles Lakers", @source="Compilation of Presidential Documents. National Basketball Association champion Los Angeles Lakers. Monday, December 13, 2010.", @president="Obama, Barack H.", @date=#<Date: 2010-12-13 ((2455544j,0s,0n),+0s,2299161j)>>, #<Presdocs::Document:0x007ff83a24b848 @id="DCPD-201000052", @city="Washington", @state="DC", @lat=nil, @lng=nil, @title="Remarks Honoring the 2009 National Basketball Association Champion Los Angeles Lakers", @source="Compilation of Presidential Documents. Addresses and Remarks. Monday, January 25, 2010.", @president="Obama, Barack H.", @date=#<Date: 2010-01-25 ((2455222j,0s,0n),+0s,2299161j)>>]
@dwillis
dwillis / seeds.json
Created March 17, 2013 23:42
NCAA Tournament overall seedings
[{"seed":1,"team":"Louisville"},{"seed":2,"team":"Kansas"},{"seed":3,"team":"Indiana"},{"seed":4,"team":"Gonzaga"},{"seed":5,"team":"Miami (FL)"},{"seed":6,"team":"Duke"},{"seed":7,"team":"Georgetown"},{"seed":8,"team":"Ohio St."},{"seed":9,"team":"New Mexico"},{"seed":10,"team":"Florida"},{"seed":11,"team":"Michigan St."},{"seed":12,"team":"Marquette"},{"seed":13,"team":"Michigan"},{"seed":14,"team":"Kansas St."},{"seed":15,"team":"Saint Louis"},{"seed":16,"team":"Syracuse"},{"seed":17,"team":"Oklahoma St."},{"seed":18,"team":"UNLV"},{"seed":19,"team":"Wisconsin"},{"seed":20,"team":"VCU"},{"seed":21,"team":"Arizona"},{"seed":22,"team":"Butler"},{"seed":23,"team":"Memphis"},{"seed":24,"team":"UCLA"},{"seed":25,"team":"Creighton"},{"seed":26,"team":"San Diego St."},{"seed":27,"team":"Notre Dame"},{"seed":28,"team":"Illinois"},{"seed":29,"team":"North Carolina"},{"seed":30,"team":"Colorado St."},{"seed":31,"team":"Pittsburgh"},{"seed":32,"team":"North Carolina St."},{"seed":33,"team":"Missouri"},{"seed":34,"tea
\n\t\t\n\t\n\t\n\t\t\t\n\t\t\n\t\t\n\t\t\thttp://www.vitter.senate.gov/newsroom/press/vitter-bp-to-pay-340-million-of-1-billion-in-early-restoration-fines\n\t\t\n\t
@dwillis
dwillis / elections.json
Last active December 17, 2015 16:59
Election Data Spec Version 2 Example
[{
"id": "md-2012-11-06-general",
"year": 2012,
"start_date": "2012-11-06",
"end_date": "2012-11-06",
"division": "ocd-division/country:us/state:md",
"result_type": "Certified",
"election_type": "General",
"party": "",
"special": false,
@dwillis
dwillis / elections.csv
Created May 24, 2013 22:25
Elections CSV Example V2
We can make this file beautiful and searchable if this error is corrected: It looks like row 2 should actually have 1 column, instead of 11 in line 1.
id year start_date end_date division results_type election_type party special offices reporting_levels vote_types notes source_url updated_at
md-2012-11-06-general 2012 11/6/12 11/6/12 ocd-division/country:us/state:md Certified General FALSE President,Senate,House,State Legislature Race-wide,County,Congressional District,State Legislative,Precinct Absentee,Provisional,Early Voting Early Voting, Absentee and Provisional vote counts are not reported at the precinct level; those counts are only included in the county level files. http://www.elections.state.md.us/elections/2012/election_data/index.html 2013-05-24 00:00:00-05:00
md-2012-04-03-primary-democratic 2012 4/3/12 4/3/12 ocd-division/country:us/state:md Certified Primary Democratic FALSE President,Senate,House,State Legislature Race-wide,County,Congressional District,State Legislative,Precinct Absentee,Provisional,Early Voting Early Voting, Absentee and Provisional vote counts are not reported at the precinct level; those counts are only included in the
@dwillis
dwillis / mccandlish.rb
Created May 26, 2013 20:25
Prototype NYT Article Search API (v2) wrapper
# Goals: allow chained methods with clear syntax
c = Mccandlish::Client.new('API_KEY')
c.query("Anthony Weiner").year(2013).location("New York City").result
# other methods include many of these: http://developer.nytimes.com/docs/read/article_search_api_v2#filters-fields