Skip to content

Instantly share code, notes, and snippets.

@gonz
Created July 26, 2011 20:20
Show Gist options
  • Save gonz/1107913 to your computer and use it in GitHub Desktop.
Save gonz/1107913 to your computer and use it in GitHub Desktop.
require 'net/http'
require 'uri'
#
#url = 'http://www.amazon.com/gp/product/B004HFS6Z0/ref=s9_simh_gw_p349_d0_i4?pf_rd_m=ATVPDKIKX0DER&pf_rd_s=center-2&pf_rd_r=1J7QTMKWJXARDQJ77VXD&pf_rd_t=101&pf_rd_p=470938631&pf_rd_i=507846'
def fetch(url)
#options.merge!(config[:http]) unless config[:http].nil?
while true do
uri = URI.parse url
options = { :use_ssl => ("https" == uri.scheme) }
puts url
Net::HTTP.start uri.host, uri.port do |http|
request = Net::HTTP::Get.new uri.request_uri, 'User-Agent' => "Pricebang/1.0"
response = http.request request
puts response.code
# Manage posible redirect
if response['Location'] == nil then
break
else
# Manage circular
url = response['Location']
end
end
end
end
response = fetch 'http://www.sears.com/shc/s/p_10153_12605_09576837000P?mv=rr'
puts response.body
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment