afa afa

My largest Sidekiq application had a memory leak and I was able to find and fix it in just few hours spent on analyzing Ruby's heap. In this post I'll show my profiling setup.

As you might know Ruby 2.1 introduced a few great changes to ObjectSpace, so now it's much easier to find a line of code that is allocating too many objects. Here is great post explaining how it's working.

I was too lazy to set up some seeding and run it locally, so I checked that test suite passes when profiling is enabled and pushed debugging to production. Production environment also suited me better since my jobs data can't be fully random generated.

So, in order to profile your worker, add the sidekiq_profiler.rb (below) to your project

Adjust number of jobs you want your worker to process before you have heap dumped.
Run a sample worker: PROFILE=1 sidekiq -C config/sidekiq.yml and wait for jobs to be processed.

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname

Common Options

-#, --progress-bar Make curl display a simple progress bar instead of the more informational standard meter.

-b, --cookie <name=data> Supply cookie with request. If no =, then specifies the cookie file to use (see -c).

-c, --cookie-jar <file name> File to save response cookies to.

	require "active_record"

	ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
	ActiveRecord::Migration.class_eval do
	create_table(:records) do \|t\|
	t.string :column
	end
	end

	data = 50_000.times.map { \|i\| Hash[column: "Column #{i}"] }

	# frozen_string_literal: true
	class Aggregated::ProcessOrgResult
	include Interactor

	delegate :object, :answer_value, :answer_max_value, :depths, to: :context

	def call
	load_params
	param = setup_param(@obj)
	sub = setup_sub(@obj)

	module FileStorage
	class BaseModel < Hashie::Trash
	include Hashie::Extensions::Coercion
	include Hashie::Extensions::MergeInitializer
	include Hashie::Extensions::IndifferentAccess
	include Hashie::Extensions::IgnoreUndeclared
	end
	end

	CREATE EXTENSION IF NOT EXISTS "unaccent"

	CREATE OR REPLACE FUNCTION slugify("value" TEXT)
	RETURNS TEXT AS $$
	-- removes accents (diacritic signs) from a given string --
	WITH "unaccented" AS (
	SELECT unaccent("value") AS "value"
	),
	-- lowercases the string
	"lowercase" AS (