Skip to content

Instantly share code, notes, and snippets.

@knowtheory
Last active December 12, 2015 06:59
Show Gist options
  • Select an option

  • Save knowtheory/4733340 to your computer and use it in GitHub Desktop.

Select an option

Save knowtheory/4733340 to your computer and use it in GitHub Desktop.
require 'open-uri'
require 'nokogiri'
require 'csv'
#######################################
#
# PHASE ONE: PAGE FETCHING
#
#######################################
# Fetch the web page into a buffer
file_handle = open('http://www.boonecountymo.org/sheriff/JailResidents/JailResidents.asp')
# read the contents of the webpage into a string
page_contents = file_handle.read
#######################################
#
# PHASE TWO: DATA EXTRACTION
#
#######################################
# Use Nokogiri to parse the string into a structured HTML object
html = Nokogiri::HTML(page_contents)
# You can now interact with the HTML object using some standard
# interfaces, including CSS selectors.
#
# This selector targets all of the rows (tr) in a particular table
selector = "html body div#canvas table.resultsTable tr"
# get all of the HTML nodes which match the selector
rows = html.css(selector)
# Using the map method, we can extract each data cell
# and assign it to a variable.
inmates = rows.map do |row|
cells = row.css("td") # get the cells from this row
cells.map{ |cell| cell.text } # get the text of each cell
end
###################################
#
# PHASE THREE: DATA STORAGE
#
###################################
# Open "inmates.csv" as a writable (that's the "w") CSV file, which we'll call "file".
CSV.open("inmates.csv", "w") do |file|
# iterate over all of the inmates, and insert the data into the file.
# this block passed to each uses curly braces ({}) instead of "do" and "end"
inmates.each{ |inmate| file << inmate }
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment