Resque memory test

average_memory.sh

for i in {1..10}
do
  RUBY_HEAP_MIN_SLOTS=974 RUBY_HEAP_SLOTS_INCREMENT=1 RUBY_HEAP_SLOTS_GROWTH_FACTOR=1 ruby test.rb &
  sleep 1
  pid=$!
  ps -o rss -p $pid | tail -1
  kill $pid
done 2> /dev/null | awk '{count+=1; sum+=$1}END{print sum/count}'

Gemfile

source 'https://rubygems.org'

gem 'json'
gem 'resque', '1.19.0'

test.rb

#!/usr/bin/env ruby

require 'rubygems'
require 'bundler'
Bundler.setup
require 'json'
require 'resque'

worker = Resque::Worker.new('test_queue')
worker.work(1)

Doing some tests to see how low I can get memory consumption using REE 2012.02 with memory tuning parameters:

RUBY_HEAP_MIN_SLOTS=974                      # How many slots - this is the lowest value that doesn't segfault
RUBY_HEAP_SLOTS_INCREMENT=1            # How many slots should be added when new slots are needed
RUBY_HEAP_SLOTS_GROWTH_FACTOR=1  # Multiplier for the increment - value of 1 achieves linear growth

Those settings should guarantee the smallest possible vm size (obviously trading off performance).

Tests:

• Nothing but loop {}: 1556k
• require 'rubygems': 2204k
• require 'bundler': 8495k
• Bundler.setup: 10882k
• require 'json': 11083k
• require 'resque': 11404k
• instantiate worker: 11413
• worker.work(1): 11462
• worker.work(1) with 10 second steady state: 11465

(Everything allows 1 second to reach steady-state unless otherwise noted)

What this all tells me is that there doesn't seem to really be a weird leak in resque, not sure what @diminish7 was seeing, but I think his memory parameters were just causing havoc.