count images on REA home ideas

home_idea_count.rb

require 'nokogiri'
require 'open-uri'

spaces = []
doc = Nokogiri::HTML(open('http://www.realestate.com.au/home-ideas'))
doc.css('nav.hi-main-nav>ul>li>a').each do |anchor|
  space = anchor.attribute('href')
  space_url = 'http://www.realestate.com.au' + space.text.sub('space', 'results') + "/list-1"
  doc_list = Nokogiri::HTML(open(space_url))
  spaces << {
    :name => space.text.sub('/home-ideas/space-', '').gsub('+', ' '),
    :space_url => space_url,
    :count => doc_list.css('.search-tools strong em').first.text.gsub(',', '').to_i
  }
end

total = spaces.inject( nil ) { |sum,space| sum ? sum + space[:count] : space[:count] }
puts "Total: #{total}"

some images may be double counted if in multiple categories

run Fri 14 Oct 2011 9:00am with a result of 2034
run Fri 14 Oct 2011 9:45am with a result of 2259

for a more accurate number, try this:-

:count => doc_list.css('.search-tools strong em').first.text.gsub(',', '').to_i

nice correction there rea-simon. that fix now implemented brings up the number to 3267 or should that be 3,267 images. That is not to say that 1 image in 2 separate "spaces" is not double counted. The point of course is that realestate.com.au has released an image gallery of less then 5,000 images with most of the images incorrectly tagged, they call it a "product" and probably have spend $500k building it ;)