Skip to content

Instantly share code, notes, and snippets.

@btaitelb
Created May 7, 2014 19:20
Show Gist options
  • Save btaitelb/9a6fdbfb7d4e0431b494 to your computer and use it in GitHub Desktop.
Save btaitelb/9a6fdbfb7d4e0431b494 to your computer and use it in GitHub Desktop.
How many recently visited domains used wordpress?
#!/usr/bin/env ruby
## Dependencies:
## * curl
## * jq (not really needed, but I like it)
## * sqlite
## * quit chrome before running this since the History file is locked
require 'uri'
HISTORY_FILE = File.expand_path("~/Library/Application Support/Google/Chrome/Default/History")
BW_KEY = "ENTER_YOUR_BUILTWITH_API_KEY"
LIMIT_TO_50 = true # 50 credits available with free key from api.builtwith.com
def domains_from_chrome(count=1000)
sql = "select url from urls order by last_visit_time desc limit #{count}"
`sqlite3 "#{HISTORY_FILE}" "#{sql}"`.split("\n").map{|u| URI.parse(u).host}.uniq
end
def built_with_wordpress?(url="builtwith.com")
lookup = "http://api.builtwith.com/v3/api.json?KEY=#{BW_KEY}&LOOKUP=#{url}"
bw_results = `curl -s "#{lookup}" | jq '.Result.Paths[].Technologies[].Name' | grep Wordpress`.strip
puts bw_results
!bw_results.empty?
end
domains = domains_from_chrome
if LIMIT_TO_50
domains = domains[0...50]
end
count = domains.count
wp_count = 0
domains.each do |domain|
print "domain: #{domain}"
wordpress = built_with_wordpress?(domain)
puts "\t wordpress? #{wordpress}\n\n"
wp_count += 1 if wordpress
end
puts "#{wp_count} / #{count} recently visited domains use wordpress"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment