Gracefully restarting Sidekiq on Elasticbeanstalk, only after it's done with the running jobs. Tested on: 64bit Amazon Linux 2016.09 v2.3.0 running Ruby 2.3 (Puma). Thanks to ssaunier for the original gist! https://gist.github.com/ssaunier/44bbebb9c0fa01953860

sidekiq.config

# Sidekiq interaction and startup script
commands:
  create_post_dir:
    command: "mkdir -p /opt/elasticbeanstalk/hooks/appdeploy/post"
    ignoreErrors: true
files:
  "/opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_sidekiq.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      . /opt/elasticbeanstalk/support/envvars

      EB_APP_DEPLOY_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
      EB_APP_PID_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_pid_dir)
      EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config container -k app_user)
      EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
      EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)

      . $EB_SUPPORT_DIR/envvars
      . $EB_SCRIPT_DIR/use-app-ruby.sh

      SIDEKIQ_PID=$EB_APP_PID_DIR/sidekiq.pid
      SIDEKIQ_CONFIG=$EB_APP_DEPLOY_DIR/config/sidekiq.yml
      SIDEKIQ_LOG=$EB_APP_DEPLOY_DIR/log/sidekiq.log

      . /opt/elasticbeanstalk/support/envvars.d/sysenv

      sleep 10

      cd $EB_APP_DEPLOY_DIR

      echo "starting sidekiq"
      su -s /bin/bash -c "bundle exec sidekiq \
        -e $RACK_ENV \
        -P $SIDEKIQ_PID \
        -C $SIDEKIQ_CONFIG \
        -L $SIDEKIQ_LOG \
        -d" $EB_APP_USER

  "/opt/elasticbeanstalk/hooks/appdeploy/pre/03_mute_sidekiq.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      . /opt/elasticbeanstalk/support/envvars

      EB_APP_DEPLOY_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
      EB_APP_PID_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_pid_dir)
      EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config container -k app_user)
      EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
      EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)

      . $EB_SUPPORT_DIR/envvars
      . $EB_SCRIPT_DIR/use-app-ruby.sh

      SIDEKIQ_PID=$EB_APP_PID_DIR/sidekiq.pid

      cd $EB_APP_DEPLOY_DIR

      if [ -f $SIDEKIQ_PID ]
      then
        echo "waiting for sidekiq to finish busy jobs"
        su -s /bin/bash -c "bundle exec rake sidekiq:wait" $EB_APP_USER
        echo "shutting down sidekiq"
        su -s /bin/bash -c "kill -TERM `cat $SIDEKIQ_PID`" $EB_APP_USER
        su -s /bin/bash -c "rm -rf $SIDEKIQ_PID" $EB_APP_USER
      fi

sidekiq.rake

namespace :sidekiq do
  require 'sidekiq/api'

  desc "Wait until 'busy' queue is finished"
  task wait: :environment do
    Sidekiq::ProcessSet.new.each(&:quiet!)
    sleep(1) until finished?
  end

  private

  def finished?
    ps = Sidekiq::ProcessSet.new
    return ps.size == 0 || ps.detect { |process| process['busy'] == 0 }
  end
end

Thanks, one question. Since the work tier in EB uses SQS, how do you handle the sidekiq configuration. Did you use the same web servers as worker servers?

Confirmed working on Passenger with Ruby 2.3 running on 64bit Amazon Linux/2.7.1 on 02/22/18

@wcpaez This configuration is for a setup in which the worker is on the same server with the web application

When running this script with multiple EC2 instances, after a deploy with Elastic Beanstalk one instance often gets stuck on quiet
Any thoughts on how to prevent this?

@andrewlynch Is your problem solved? If it's solved, can you tell me how you did it?

@andrewlynch Did you put the files in the right directories? This isn't supposed to happen because sidekiq:wait is executed pre-deploy
I'm no longer working with Elasticbeanstalk so I'm not sure if it's working the same way it used to

@maxdbn sidekiq/sidekiq#4336

@rokumatsumoto If I remember correctly I only quiet the processes so they won't get any new jobs so I can restart sidekiq without losing anything. When it's restarted they should work as usual, the issue you sent is not relevant :)

Confirmed working on Passenger with Ruby 2.6.0 running on 64bit Amazon Linux/2.7.1 on 23/11/19.

thanks

This script only works when you have a single worker. If you have a multiple workers with rolling deployments, it will fail. It will deploy 1 server at a time and every time it will quite all of the process even which are already deployed which causes the issue mentioned by andrewlynch .