Skip to content

Instantly share code, notes, and snippets.

@aosalias
Created May 12, 2016 23:27
Show Gist options
  • Save aosalias/cccd64d9ce472f973b61ce02b9821311 to your computer and use it in GitHub Desktop.
Save aosalias/cccd64d9ce472f973b61ce02b9821311 to your computer and use it in GitHub Desktop.
Scrape store info from schnucks website
require 'mechanize'
require 'CSV'
require 'byebug'
robot = Mechanize.new
schnucks = robot.get 'http://www.schnucks.com/stores/'
CSV.open('./schnucks_data.csv', 'wb') do |csv|
csv << %w(Name Street City State Zip Phone ID)
schnucks.links_with(href: /stores\/store\.asp/).each do |link|
begin
name = link.text.strip.match(/(.*)/)[1]
page = link.click
address= page.at("span[itemprop='address']").children
street = address[0].text.strip
city, state, zip = *address[2].text.strip.split(' ')
id = page.uri.query.split('=')[1]
phone = page.at("span[itemprop='telephone']").children.text
csv << [name, street, city, state, zip, phone, id]
rescue Exception => e
debugger
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment