mkcode

Overview
--------
Initial state: object count: 354826
Memory allocated outside heap (bytes): 55417649

GC Stats:
--------
count : 46
heap_allocatable_pages : 118

mkcode

Total allocated: 8509168 bytes (74050 objects)
Total retained:  256884 bytes (1755 objects)

allocated memory by gem
-----------------------------------
   3035195  sprockets-3.5.2
   2549711  2.3.0/lib
   1145461  binding_of_caller-0.7.2
    604629  activesupport-4.2.5.2
    375267  actionview-4.2.5.2

mkcode

ObjectSpace stats:

TOTAL: 747936
FREE: 280189
T_STRING: 174345
T_ARRAY: 114800
T_IMEMO: 80222
T_HASH: 61297
T_DATA: 10980
T_OBJECT: 10472

mkcode

Overview
--------
Initial state: object count: 181531
Memory allocated outside heap (bytes): 29931291

GC Stats:
--------
count : 40
heap_allocatable_pages : 11

mkcode

Total allocated: 8512525 bytes (74080 objects)
Total retained:  255630 bytes (1749 objects)

allocated memory by gem
-----------------------------------
   3037410  sprockets-3.5.2
   2550216  2.3.0/lib
   1143973  binding_of_caller-0.7.2
    604899  activesupport-4.2.5.2
    375267  actionview-4.2.5.2

mkcode

ObjectSpace stats:

TOTAL: 1142492
FREE: 669062
T_STRING: 179003
T_ARRAY: 114638
T_IMEMO: 80083
T_HASH: 61165
T_OBJECT: 12105
T_DATA: 10860

mkcode

<%
  # Quick function to calculate the correct number DB_POOL size. DB_POOL size is
  # reused for the puma web server and sidekiq. It cannot be less than the
  # amount the concurrency of either.

  db_needed_for_web     = ENV.fetch('WEB_CONCURRENCY', 1) * ENV.fetch('WEB_THREADS', 1)
  db_needed_for_sidekiq = ENV.fetch('SIDEKIQ_CONCURRENCY', 0)

  # The minimum pool size is the maximum of the sidekiq and web concurrency. We
  # add some extras to the pool for sidekiq's scheduler and one off dynos.

mkcode

  (defvar-local prose-proselint-enabled nil
    "Buffer local proselint enabled status")

  (defun prose-proselint-enabled-p ()
    "Workaround for proselint flycheck checker predicate"
    prose-proselint-enabled)

  (flycheck-define-checker proselint
    "A linter for prose."
    :command ("proselint" source-inplace)

mkcode

My largest Sidekiq application had a memory leak and I was able to find and fix it in just few hours spent on analyzing Ruby's heap. In this post I'll show my profiling setup.

As you might know Ruby 2.1 introduced a few great changes to ObjectSpace, so now it's much easier to find a line of code that is allocating too many objects. Here is great post explaining how it's working.

I was too lazy to set up some seeding and run it locally, so I checked that test suite passes when profiling is enabled and pushed debugging to production. Production environment also suited me better since my jobs data can't be fully random generated.

So, in order to profile your worker, add this to your Sidekiq configuration:

if ENV["PROFILE"]

mkcode

To friends in NYC - Coronavirus

March 4, 2020

This document is an appeal for friends and family in NYC to begin behaving as if 'Community Spread' of the corona virus is currently happening. We now have many examples throughout the world which show that early dramatic action is most effective in curtailing the spread of virus. Simple prudent actions now, can have big impacts.

Appropriate actions given Community Spread

Please continue to follow all of the standard advice you have heard regarding thoroughly washing hands, not touching your face, and avoiding body contact with others. In addition to this, please begin voluntarily practicing 'Social Distancing' techniques. These include: