Skip to content

Instantly share code, notes, and snippets.

@jaimeiniesta
Created February 18, 2014 11:14
Show Gist options
  • Save jaimeiniesta/9069017 to your computer and use it in GitHub Desktop.
Save jaimeiniesta/9069017 to your computer and use it in GitHub Desktop.
MetaInspector example script
******************************************************************
URL: https://github.com/
TITLE: GitHub · Build software better, together.
KEYWORDS:
DESCRIPTION: Build software better, together.
******************************************************************
******************************************************************
URL: https://delicious.com/
TITLE: Delicious
KEYWORDS: del.icio.us, delicious, bookmarks, bookmark, social bookmarking, links, link, stacks, stack
DESCRIPTION: Keep, share, and discover the best of the Web using Delicious, the world's leading social bookmarking service.
******************************************************************
******************************************************************
URL: http://www.ifixit.com/
TITLE: iFixit: The free repair manual
KEYWORDS: iFixit, Mac, MacBook, iPod, iPhone, Repair, Parts, XBox, PlayStation, Game Boy, Nintendo DSi, tools
DESCRIPTION: Repair your electronics yourself. iFixit is the repair manual you can edit. We sell tools, parts and upgrades for Apple Mac, iPod, iPhone, iPad, and MacBook as well as game consoles.
******************************************************************
# This example script will scrape the URLs on urls.txt
# and write the results on data.txt
#
# Tested on ruby 2.1.0, metainspector 2.1.0
#
require 'metainspector'
File.open('data.txt', 'w') do |f|
File.open('urls.txt', 'r').each do |url|
page = MetaInspector.new(url, allow_redirections: :all)
f.puts "******************************************************************"
f.puts "URL: #{page.url}"
f.puts "TITLE: #{page.title}"
f.puts "KEYWORDS: #{page.meta['keywords']}"
f.puts "DESCRIPTION: #{page.meta['description']}"
f.puts "******************************************************************\n\n"
end
end
http://github.com
http://delicious.com
http://www.ifixit.com
@sarahbkim
Copy link

Hey, I finally got a chance to take a look at this, thanks!

Works perfectly. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment