Created
January 6, 2010 17:47
-
-
Save house9/270459 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
require 'net/http' | |
require 'uri' | |
def get_html_content(requested_url) | |
url = URI.parse(requested_url) | |
full_path = (url.query.blank?) ? url.path : "#{url.path}?#{url.query}" | |
the_request = Net::HTTP::Get.new(full_path) | |
the_response = Net::HTTP.start(url.host, url.port) { |http| | |
http.request(the_request) | |
} | |
raise "Response was not 200, response was #{the_response.code}" if the_response.code != "200" | |
return the_response.body | |
end | |
# this will fail with ArgumentError: HTTP request path is empty | |
s = get_html_content("http://www.google.com") | |
# these should be fine | |
s = get_html_content("http://www.google.com/") | |
s = get_html_content("http://github.com/search?q=http") | |
# above code does not handle redirects but raises exception for non-200 | |
s = get_html_content("http://www.yahoo.com/") # http 302 | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment