Last active
September 14, 2019 05:13
-
-
Save sulmanweb/a5e4e3b15653137d19b4db1b1555a324 to your computer and use it in GitHub Desktop.
Web Scraping Script in Ruby
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
require 'HTTParty' | |
require 'Nokogiri' | |
class RubygemsScrapper | |
attr_accessor :parse_page | |
# initialize repo for ruby gems requires query string | |
def initialize(q) | |
doc = HTTParty.get("https://rubygems.org/search?query=#{q}") | |
@parse_page ||= Nokogiri::HTML(doc) | |
end | |
# get the first result's version or if not found returns -1 | |
def get_latest_version | |
begin | |
parse_page.css('.gems__gem').css('.gems__gem__version').children[0].text | |
rescue | |
-1 # Not found | |
end | |
end | |
# get the first result's link to ruby gems org or if not found returns -1 | |
def get_link | |
begin | |
"https://rubygems.org" + parse_page.css('.gems__gem').attribute('href').value | |
rescue | |
-1 # not found | |
end | |
end | |
# Calling scrapper | |
scrapper = RubygemsScrapper.new('nokogiri') | |
p scrapper.get_latest_version | |
p scrapper.get_link | |
end |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment