Laravel5 Queue + ElasticBeanstalk + Supervisord (http://www.lifeofguenter.de/2015/04/laravel-queues-with-supervisor-on.html)

supervise.config

# supervisor
#
# Author: Günter Grodotzki <[email protected]>
# Version: 2015-04-25
#
# - set param "SUPERVISE=enable" to activate
# - chmod +x supervise.sh
#
packages:
  yum:
    python27-setuptools: []
container_commands:
  01-supervise:
    command: ".ebextensions/supervise.sh"

supervise.sh

#!/bin/bash
#
# Author: Günter Grodotzki ([email protected])
# Version: 2015-04-25
#
# install supervisord
#
# See:
# - https://github.com/Supervisor/initscripts
# - http://supervisord.org/

if [ "${SUPERVISE}" == "enable" ]; then
  
  export HOME="/root"
  export PATH="/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin"
  
  easy_install supervisor
    
  cat <<'EOB' > /etc/init.d/supervisord
#!/bin/bash
#
# supervisord   Startup script for the Supervisor process control system
#
# Author:       Mike McGrath <[email protected]> (based off yumupdatesd)
#               Jason Koppe <[email protected]> adjusted to read sysconfig,
#                   use supervisord tools to start/stop, conditionally wait
#                   for child processes to shutdown, and startup later
#               Erwan Queffelec <[email protected]>
#                   make script LSB-compliant
#
# chkconfig:    345 83 04
# description: Supervisor is a client/server system that allows \
#   its users to monitor and control a number of processes on \
#   UNIX-like operating systems.
# processname: supervisord
# config: /etc/supervisord.conf
# config: /etc/sysconfig/supervisord
# pidfile: /var/run/supervisord.pid
#
### BEGIN INIT INFO
# Provides: supervisord
# Required-Start: $all
# Required-Stop: $all
# Short-Description: start and stop Supervisor process control system
# Description: Supervisor is a client/server system that allows
#   its users to monitor and control a number of processes on
#   UNIX-like operating systems.
### END INIT INFO

# Source function library
. /etc/rc.d/init.d/functions

# Source system settings
if [ -f /etc/sysconfig/supervisord ]; then
    . /etc/sysconfig/supervisord
fi

# Path to the supervisorctl script, server binary,
# and short-form for messages.
supervisorctl=${SUPERVISORCTL-/usr/bin/supervisorctl}
supervisord=${SUPERVISORD-/usr/bin/supervisord}
prog=supervisord
pidfile=${PIDFILE-/var/run/supervisord.pid}
lockfile=${LOCKFILE-/var/lock/subsys/supervisord}
STOP_TIMEOUT=${STOP_TIMEOUT-60}
OPTIONS="${OPTIONS--c /etc/supervisord.conf}"
RETVAL=0

start() {
    echo -n $"Starting $prog: "
    daemon --pidfile=${pidfile} $supervisord $OPTIONS
    RETVAL=$?
    echo
    if [ $RETVAL -eq 0 ]; then
        touch ${lockfile}
        $supervisorctl $OPTIONS status
    fi
    return $RETVAL
}

stop() {
    echo -n $"Stopping $prog: "
    killproc -p ${pidfile} -d ${STOP_TIMEOUT} $supervisord
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && rm -rf ${lockfile} ${pidfile}
}

reload() {
    echo -n $"Reloading $prog: "
    LSB=1 killproc -p $pidfile $supervisord -HUP
    RETVAL=$?
    echo
    if [ $RETVAL -eq 7 ]; then
        failure $"$prog reload"
    else
        $supervisorctl $OPTIONS status
    fi
}

restart() {
    stop
    start
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        status -p ${pidfile} $supervisord
        RETVAL=$?
        [ $RETVAL -eq 0 ] && $supervisorctl $OPTIONS status
        ;;
    restart)
        restart
        ;;
    condrestart|try-restart)
        if status -p ${pidfile} $supervisord >&/dev/null; then
          stop
          start
        fi
        ;;
    force-reload|reload)
        reload
        ;;
    *)
        echo $"Usage: $prog {start|stop|restart|condrestart|try-restart|force-reload|reload}"
        RETVAL=2
esac

exit $RETVAL
EOB

  chmod +x /etc/init.d/supervisord

  cat <<'EOB' > /etc/sysconfig/supervisord
# Configuration file for the supervisord service
#
# Author: Jason Koppe <[email protected]>
#             orginal work
#         Erwan Queffelec <[email protected]>
#             adjusted to new LSB-compliant init script

# make sure elasticbeanstalk PARAMS are being passed through to supervisord
. /opt/elasticbeanstalk/support/envvars

# WARNING: change these wisely! for instance, adding -d, --nodaemon
# here will lead to a very undesirable (blocking) behavior
#OPTIONS="-c /etc/supervisord.conf"
PIDFILE=/var/run/supervisord/supervisord.pid
#LOCKFILE=/var/lock/subsys/supervisord.pid

# Path to the supervisord binary
SUPERVISORD=/usr/local/bin/supervisord

# Path to the supervisorctl binary
SUPERVISORCTL=/usr/local/bin/supervisorctl

# How long should we wait before forcefully killing the supervisord process ?
#STOP_TIMEOUT=60

# Remove this if you manage number of open files in some other fashion
#ulimit -n 96000
EOB

  mkdir -p /var/run/supervisord/
  chown webapp: /var/run/supervisord/
  
  cat <<'EOB' > /etc/supervisord.conf
[unix_http_server]
file=/tmp/supervisor.sock
chmod=0777

[supervisord]
logfile=/var/app/support/logs/supervisord.log
logfile_maxbytes=0
logfile_backups=0
loglevel=warn
pidfile=/var/run/supervisord/supervisord.pid
nodaemon=false
nocleanup=true
user=webapp

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock

[program:laravel_queue]
command=php artisan queue:listen
directory=/var/app/current
stdout_logfile=/var/app/support/logs/laravel-queue.log
logfile_maxbytes=0
logfile_backups=0
redirect_stderr=true
autostart=true
autorestart=true
startretries=86400
EOB

  # this is now a little tricky, not officially documented, so might break but it is the cleanest solution
  # first before the "flip" is done (e.g. switch between ondeck vs current) lets stop supervisord
  echo -e '#!/usr/bin/env bash\nservice supervisord stop' > /opt/elasticbeanstalk/hooks/appdeploy/enact/00_stop_supervisord.sh
  chmod +x /opt/elasticbeanstalk/hooks/appdeploy/enact/00_stop_supervisord.sh
  # then right after the webserver is reloaded, we can start supervisord again
  echo -e '#!/usr/bin/env bash\nservice supervisord start' > /opt/elasticbeanstalk/hooks/appdeploy/enact/99_z_start_supervisord.sh
  chmod +x /opt/elasticbeanstalk/hooks/appdeploy/enact/99_z_start_supervisord.sh
fi

Thank you so much for this. I struggled with getting this going – I could not figure out why my queue ran fine when I ran it myself (after ssh-ing into the box), but was so quiet on start-up. The line that enlightened me was this:

# make sure elasticbeanstalk PARAMS are being passed through to supervisord
. /opt/elasticbeanstalk/support/envvars

@russmatney , just trying the script above on my elasticbeanstalk i didnt see the service running on my servers
which param that i should put in on ELB ?

able to run the script just adding the param in elasticbeanstalk
see the image below

thx you very much sir

Awesome library, thank you!

It is informative! Thank you!

Hi , i need some help here, i created an elasticbeanstalk application in laravel in one environment , than i created another environment as worker and hosted the above scripts in the worker with the SUPERVISE environment variable. Than i changed by laravel app to use sqs and the queue created by the worker.

Now i send the email from the main application using mail::queue method but nothing happens , i can see sometimes they are on the queue but they are never fired from the queue.

Can any body help me to understand what i have missed here.

@coolneo I have the same issue as yours. May i know whether you can solve your issue?

When i run this in my app,i receive this:

[Instance: i-07352797c0c2cd44f] Command failed on instance. Return code: 126 Output: /bin/sh: .ebextensions/supervise.sh: Permission denied. container_command 01-supervise in .ebextensions/supervise.config failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.

Dont think this is working. Can someone pls help?

after command: "chmod +x ./ebextensions/supervise.config" i received the following error:

[Instance: i-07352797c0c2cd44f] Command failed on instance. Return code: 126 Output: /bin/sh: .ebextensions/supervise.sh: /bin/bash^M: bad interpreter: No such file or directory. container_command 02-supervise in .ebextensions/supervise.config failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.

@coolneo - This script DOES NOT require a Elastic Beanstalk worker environment. This script makes sure the Laravel application is constantly pulling messages from the queue on ANY Elastic Beanstalk Environment. So you can just run this on a web server elastic beanstalk environment and it's fine.

Make sure you create the SUPERVISE environment variable for the elastic beanstalk environment. This value should be set to enable. Without this, it will not run.

@laics1984 - Can you run that command by SSHing into the server directly? ./ebextensions/supervise.sh.

Why are you using the command 02-supervise, the example above just has this...

container_commands:
  01-supervise:
    command: ".ebextensions/supervise.sh"

It looks like you modified it. Make sure all of the file names are correct if you're changing the names of files or functions.

supervisor.sh is not working despite I am not getting errors in eb-activity log. I have followed instructions carefully. Please help me to make it work

@laics1984 - check the line endings of your supervise.sh file, it must be LF. You'll get the /bin/bash^M: bad interpreter: No such file or directory. error if your line endings are anything but LF.

And I think you mean chmod +x ./ebextensions/supervise.sh rather than chmod +x ./ebextensions/supervise.config as it's the bash script that is failing to execute, not the config file...

Works like a charm! Thank you for sharing this, very very kind sir!!

This is awesome! Worked like a charm. Thanks, Guenter!

For future peeps who ran into Permission Denied error, you need to do this in supervise.config

# supervisor
#
# Author: Günter Grodotzki <[email protected]>
# Version: 2015-04-25
#
# set param "SUPERVISE=enable" to activate
#
packages:
  yum:
    python27-setuptools: []
container_commands:
  01-supervise:
    command: "chmod +x .ebextensions/supervise.sh"
  02-supervise:
     command: ".ebextensions/supervise.sh"

Need to ensure proper permission is set for the bash script

Does this work for Laravel 7.0? I tried it and I am getting the error
[2020-03-07T16:08:47.319Z] ERROR [30949] : Command execution failed: Activity failed. (ElasticBeanstalk::ActivityFatalError) caused by: /bin/sh: .ebextensions/supervise.sh: Permission denied (ElasticBeanstalk::ExternalInvocationError)
I followed all the steps carefully