README.textile

This is my modest attempt at a key value store. I use the filesystem to store any object or data requested to be set. `runner` will give you a demonstration of how it works. Right now, the folder splitting depth is 1 so a key like apple would be in /a/apple.

I included a `benchmark` as I’d be curious as to how this entry performs compared to others. Even then my benchmark is far from exhaustive. My keys in the test are random, a better test would see what impact grouping keys together would have. eg (aa, ab, ac). FsStore may also perform well with smaller numbers of keys, but when the number of keys gets to be over a certain size, I can see there being a significant performance hit as the OS tries to find one file in a directory of 10000. I also noticed on my system, which is a mac, that creating 10000 keys at once, makes mds get really busy, I should probably put the data store in a folder with .noindex appended so mds leaves it alone.

benchmark.rb

require 'fs_store'

letters = ("a".."z").to_a
count = 10000
keys = []
heading "generating 1000 random keys"
count.times do
  key = ""
  10.times { key << letters[rand(25)] }
  keys << key
end

f = FsStore.new("/tmp/random")

puts start_time = Time.now
keys.each do |key|
  f.set(key, "blah")
end
keys.each do |key|
  f.get(key)
end
puts end_time = Time.now

puts end_time - start_time

fs_store.rb

require 'fileutils'
require 'benchmark'

class FsStore
  def initialize(path)
    self.base = path
  end
  
  def get(key)
    path = path_for_key(key)
    Marshal.load(File.read(path))
  end

  def set(key, value)
    path = path_for_key(key)
    
    begin
      f = File.open(path, "w+")
      f.print Marshal.dump(value)
    ensure
      f.close
    end
    value
  end
  
  def list
    path = File.join(base, "*", "*")
    Dir.glob(path).map do |dir|
      dir.split("/").last
    end
  end
  
  def delete(key)
    path = path_for_key(key)
    File.delete(path)
  end
  
  private

  def path_for_key(key)
    key_string = key.to_s
    [base_path_for_key(key_string), key_string].join("/")
  end

  def base_path_for_key(key)
    key_string = key.to_s
    path = [base, key_string[0].chr].join("/")
    FileUtils.mkdir_p(path)
    path
  end
  
  def base=(path)
    FileUtils.mkdir_p(path)
    @base = path
  end
  
  def base
    @base
  end
end

runner.rb

require 'fs_store'

def heading(text)
  puts "===#{text}==="
end

heading "initialize the store with a path"
f =  FsStore.new("/tmp/data2")

heading "set some keys"
f.set("hello", "world")
f.set(:foo, "bar")

heading "retrieve those keys with a symbol or string"
puts f.get(:hello)
puts f.get("foo")

heading "listing our objects"
puts f.list

heading "store objects"
f.set("time", Time.now)
puts f.get("time").class

heading "delete keys"
f.delete("hello")