read times for large key values - Memcache and Redis

read-times-large-values.markdown

Memcache

Server: Ubuntu 16.04.1
Client: Ubuntu 16.04.1, Ruby 2.3 client (Dalli)
Memcache version: 1.4.25
Config changes:

• Changed bind setting to allow external connections

Client code:

URL="" # server url  
require 'dalli'  
require 'benchmark'  

def measure_(data)  
  options = { value_max_bytes: 100_000_000 }  
  dc = Dalli::Client.new(URL, options)  
  dc.set 'xyz', data  
  Benchmark.measure{ dc.get 'xyz' }.total  
end  

Observations:

d1 = File.read("data-7.9mb")  # 7.9mb  
d2 = File.read("data-16mb")  # 16mb  
d3 = File.read("data-95mb")  # 95mb  

measure_(d1) #=> 0.07, 0.11  
measure_(d2) #=> 0.18, 0.14, 0.27
measure_(d3) #=> 1.0, 0.97

Redis

Server: Ubuntu 14.04, with Digital Ocean Redis - http://do.co/redisapp
Client: Ubuntu 16.04.1, Ruby 2.3, redis gem
Redis version: 3.2.1
Config changes:

• Added authentication
• Changed bind setting to allow external connections
• Disabled persistence

Client code:

URL = ""
PASS = ""
require 'redis'
require 'benchmark'
def measure_(data)
  r = Redis.new(host: url, port: 6379, password: pass)
  r.set 'abc', data
  Benchmark.measure{ r.get 'abc' }
end

Observations:

• For client on a separate machine, the time taken is sygnificantly different
• Data sized 256kb and above times-out (60s). Upto 128k is okay
• For the same machine, there is no timeout even for a 1mb data. 8mb takes a few seconds, but goes through