Skip to content

Instantly share code, notes, and snippets.

@knowtheory
Created February 14, 2011 06:39
Show Gist options
  • Select an option

  • Save knowtheory/825563 to your computer and use it in GitHub Desktop.

Select an option

Save knowtheory/825563 to your computer and use it in GitHub Desktop.
require 'dm-core'
require 'dm-validations'
require 'dm-aggregates'
require 'dm-types'
require 'dm-migrations'
# See chromium/src/base/{time.h,time_mac.cc}
begin
# provide Time#utc_time for DateTime#to_time in AS
require 'active_support/core_ext/time/calculations'
rescue LoadError
# do nothing, extlib is being used and does not require this method
end
module DataMapper
class Property
class ChromeEpochTime < Integer
def custom?; true; end # hack for now to address problems with the property system in dm-core
def load(value)
return value unless value.respond_to?(:to_int)
::Time.at((value.to_i/10**6)-11644473600)
end
def dump(value)
case value
when ::Integer, ::Time then (value.to_i + 11644473600) * 10**6
when ::DateTime then (value.to_time.to_i + 11644473600) * 10**6
end
end
end # class EpochTime
end # class Property
end # module DataMapper
DataMapper.setup(:default, "sqlite::memory:")
class Visit
include DataMapper::Resource
property :id, Serial
property :url_id, Integer, :required => true, :field => "url"
property :visit_time, ChromeEpochTime, :required => true
belongs_to :url, :child_key => [:url_id]
end
class Url # CREATE TABLE urls(
include DataMapper::Resource
property :id, Serial # id INTEGER PRIMARY KEY,
property :url, URI # url LONGVARCHAR,
property :title, String # title LONGVARCHAR,
has n, :visits
end # );
DataMapper.auto_migrate!
visit_data = [{:id=>385231, :url_id=>178863, :visit_time=>"2011-02-12 08:31:17 -0500"}, {:id=>385232, :url_id=>165351, :visit_time=>"2011-02-12 08:31:17 -0500"}, {:id=>385233, :url_id=>160382, :visit_time=>"2011-02-12 08:31:17 -0500"}, {:id=>385234, :url_id=>164510, :visit_time=>"2011-02-12 08:31:17 -0500"}, {:id=>385235, :url_id=>174415, :visit_time=>"2011-02-12 08:31:17 -0500"}, {:id=>385236, :url_id=>178693, :visit_time=>"2011-02-12 08:31:17 -0500"}, {:id=>385237, :url_id=>176342, :visit_time=>"2011-02-12 08:31:18 -0500"}, {:id=>385238, :url_id=>171670, :visit_time=>"2011-02-12 08:31:18 -0500"}, {:id=>385239, :url_id=>162600, :visit_time=>"2011-02-12 08:31:18 -0500"}, {:id=>385240, :url_id=>163774, :visit_time=>"2011-02-12 08:31:20 -0500"}]
url_data = [{:id=>160382, :url=>"http://www.esquire.com/print-this/roger-ailes-0211?page=all", :title=>"Print - Why Does Roger Ailes Hate America? - Esquire"}, {:id=>162600, :url=>"http://blogs.agu.org/outdoorscience/2011/01/26/why-are-female-science-writers-invisible/", :title=>"AGU Blogosphere | Outdoor Science | Why are female science writers invisible?"}, {:id=>163774, :url=>"http://www.propublica.org/article/loan-mod-program-crippled-by-lax-oversight-and-deference-to-banks", :title=>"Govt's Loan Mod Program Crippled by Lax Oversight and Deference to Banks - ProPublica"}, {:id=>164510, :url=>"http://arstechnica.com/tech-policy/news/2011/01/openleaks-launches-website-and-mission-but-platform-still-in-progress.ars?utm_source=rss&utm_medium=rss&utm_campaign=rss", :title=>"WikiLeaks alternative OpenLeaks goes live"}, {:id=>165351, :url=>"http://matt.might.net/articles/red-black-delete/", :title=>""}, {:id=>171670, :url=>"http://i50.tinypic.com/2lj71c7.png", :title=>"2lj71c7.png (894x909)"}, {:id=>174415, :url=>"http://knowtheory.panel.videojuicer.com/assets", :title=>"Assets - Videojuicer"}, {:id=>176342, :url=>"http://railsrx.com/2011/02/10/text-and-mate/", :title=>"Text And Mate « Rails Test Prescriptions Blog"}, {:id=>178693, :url=>"http://knowtheory.panel.videojuicer.com/login", :title=>"Log in to seed KNOWTHEORY - Videojuicer"}, {:id=>178863, :url=>"http://volnitsky.com/project/str_search/", :title=>"Substring search algorithm"}]
visit_data.map{ |v| visit = Visit.create(:url_id=>v[:url_id], :visit_time=>DateTime.parse(v[:visit_time])); visit.errors }
url_data.each{ |u| Url.create(u) }
visits = Visit.all('visit_time.gte' => DateTime.now.midnight.prev_day.prev_day, 'visit_time.lt' => DateTime.now.midnight.prev_day)
urls = visits.map(&:url)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment