Skip to content

Instantly share code, notes, and snippets.

@Narnach
Last active August 29, 2015 13:57
Show Gist options
  • Select an option

  • Save Narnach/9601882 to your computer and use it in GitHub Desktop.

Select an option

Save Narnach/9601882 to your computer and use it in GitHub Desktop.
A more efficient way to store data in the DB than the default approach of doing a raw Base64 on the Marshal-ed version of a session hash.
# Add this to your Gemfile:
#
# # Session store built on top of ActiveRecord
# gem 'activerecord-session_store', github: 'rails/activerecord-session_store'
# # C-based parser backend for multi_json
# gem 'oj'
# # Unified API for parsing JSON
# gem 'multi_json'
#
class Session < ActiveRecord::SessionStore::Session
def self.find_by_session_id(session_id)
self.where(session_id: session_id).first
end
def self.cleanup(threshold=1.week.ago)
where(['updated_at <= ?', threshold]).delete_all
end
# The default marshaling behavior does a base64 encode of the output of Marshal.dump.
# There are two problems: the stored data uses up quite a lot of space
# Here are the numbers from a test session with quite some data in it:
#
# * The default Session.marshal would have needed 2973 bytes.
# * Now we need only 785 bytes to store 1616 bytes of json (before base64, gzipped json is 579 bytes)
#
# This saves us 46% storage space + less time to read them from disk + less time to transfer them across the wire.
#
# A benchmark (10k operations with GC disabled) resulted in this:
#
# Custom Marshal: 0.065ms
# Custom Unmarshal: 0.062ms
# Default Marshal: 0.094ms
# Default Unmarshal: 0.069ms
#
# A 31% time savings for marshaling and 10% save for unmarshaling is nice.
#
# Since Mysql 5.0.5 we can have varchars larger than 256 bytes (max is 16k), so we only use as many bytes for storage as we need.
# We could have skipped our own base64 encoding step to gain better storage compression and to save even more time processing the data.
# Unfortunately Rails does not support the varbinary type by default. If we feel we need the speed boost, we can always build it later.
def self.marshal(data)
return unless data
json = MultiJson.dump(data)
compressed_json = ActiveSupport::Gzip.compress(json)
base64_compressed_json = Base64.encode64(compressed_json)
base64_compressed_json
end
def self.unmarshal(base64_compressed_json)
return unless base64_compressed_json
compressed_json = Base64.decode64(base64_compressed_json)
json = ActiveSupport::Gzip.decompress(compressed_json)
data = MultiJson.load(json)
data
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment