Installing Supervisor on Elastic Beanstalk - 2021 - Linux AMI 2 - Laravel Worker Setup

readme.md

• Create the following folder structure in your root project directory: .ebextensions/supervisor

• Place the supervisor.config under .ebextensions/

• Place the setup.sh under .ebextensions/supervisor/

• Run "chmod +x .ebextensions/supervisor/setup.sh"

• Place the supervisord.conf under .ebextensions/supervisor/

• Place the supervisor_laravel.conf under .ebextensions/supervisor/

Feel free to update the files according to your needs. You mostly will need to change the command inside the supervisor_laravel.conf file

If you want to check if the supervisor is running, connect to the instance and run "supervisorctl status". If you want stop/kill supervisor run "ps aux | grep "[/]usr/bin/supervisord", get the PID and then run "kill PID_NUMBER"

setup.sh

#!/bin/bash

# If you want to have more than one application, and in just one of them to run the supervisor, uncomment the lines below, 
# and add the env variable IS_WORKER as true in the EBS application you want the supervisor

#if [ "${IS_WORKER}" != "true" ]; then
#    echo "Not a worker. Set variable IS_WORKER=true to run supervisor on this instance"
#    exit 0
#fi

echo "Supervisor - starting setup"
. /opt/elasticbeanstalk/deployment/env

if [ ! -f /usr/bin/supervisord ]; then
    echo "installing supervisor"
    easy_install supervisor
else
    echo "supervisor already installed"
fi

if [ ! -d /etc/supervisor ]; then
    mkdir /etc/supervisor
    echo "create supervisor directory"
fi

if [ ! -d /etc/supervisor/conf.d ]; then
    mkdir /etc/supervisor/conf.d
    echo "create supervisor configs directory"
fi

. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisor_laravel.conf > /etc/supervisor/conf.d/supervisor_laravel.conf

if ps aux | grep "[/]usr/bin/supervisord"; then
    echo "supervisor is running"
else
    echo "starting supervisor"
    /usr/bin/supervisord
fi

/usr/bin/supervisorctl reread
/usr/bin/supervisorctl update

echo "Supervisor Running!"

supervisor.config

container_commands:
    01_install_supervisor:
        command: ".ebextensions/supervisor/setup.sh"

supervisor_laravel.conf

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /var/app/current/artisan queue:work sqs --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
;user=forge
numprocs=3
redirect_stderr=true
stderr_logfile=/var/log/supervisor_laravel.err.log
stdout_logfile=/var/log/supervisor_laravel.out.log
stopwaitsecs=3600

supervisord.conf

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10           ; (num of main logfile rotation backups;default 10)
loglevel=info                ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false               ; (start in foreground if true;default false)
minfds=1024                  ; (min. avail startup file descriptors;default 1024)
minprocs=200                 ; (min. avail process descriptors;default 200)
environment=SYMFONY_ENV=prod

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

[include]
files = /etc/supervisor/conf.d/*.conf

[inet_http_server]
port = 9000
username = user
password = pw

starting supervisor
2023-01-23 09:32:11,528 P3636 [INFO] /usr/lib/python2.7/site-packages/supervisor-4.2.5-py2.7.egg/supervisor/options.py:475: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
2023-01-23 09:32:11,528 P3636 [INFO] 'Supervisord is running as root and it is searching '
2023-01-23 09:32:11,528 P3636 [INFO] Error: No config file found at default paths (/usr/etc/supervisord.conf, /usr/supervisord.conf, supervisord.conf, etc/supervisord.conf, /etc/supervisord.conf, /etc/supervisor/supervisord.conf); use the -c option to specify a config file at a different path
2023-01-23 09:32:11,528 P3636 [INFO] For help, use /usr/bin/supervisord -h

facing same issue anyone got it fixed?

It worked for me. Combining the solutions from @johnyvelho @omidnadeem @johnabelardom, it works fine. Thanks a lot :)

@adityar15 Can you please show us your complete working version?

can you share your working version of files in .ebextensions

If someone is struggling with the error "/bin/sh: .ebextensions/supervisor/setup.sh: /bin/bash^M: bad interpreter: No such file or directory" is because you're uploading using eb deploy from windows and the ZIP file encodes newlines in windows format (CRLF), note the "^M" after "bash".

To fix that, I've added new commands to supervisor.config

container_commands:
    01_fix_script_permissions:
        command: "chmod +x .ebextensions/supervisor/setup.sh"
    02_fix_newlines_a:
        command: "sed -i 's/^M$//' .ebextensions/supervisor/setup.sh"
    02_fix_newlines_b:
        command: "sed -i $'s/\r$//' .ebextensions/supervisor/setup.sh"
    03_install_supervisor:
        command: ".ebextensions/supervisor/setup.sh"

The first command adds the execution bit to the file in case it doesn't have it. The second (both a and b) removes those windows newlines (sincerely I don't know wich one fixes it, but I've left both and it works).

Question about this three lines in setup.sh (31,32,33):

. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisor_laravel.conf > /etc/supervisor/conf.d/supervisor_laravel.conf

Why does all three start with . /opt/elasticbeanstalk/deployment/env && ?

I suppose they try to load the environment variables, but for what purpose? Is it neccessary?
It's throwing me an error (bash: shell: command not found) and I've removed that part (now all three start at cat ...) and works fine.

above all .ebextenstion files have succesfully deployed to beanstalk.But i no log files (supervisor_laravel.err.log, supervisor_laravel.out.log) have created .Is that permission problem ?? if so means how can i add permisssion commands ?

@aliGIThaider @collisiondetection @adityar15 @ashiquepclilac I created a git repo for the sample bundle I used in my project: https://github.com/johnabelardom/laravel-awseb-bundle

@aliGIThaider @collisiondetection @adityar15 @ashiquepclilac I created a git repo for the sample bundle I used in my project: https://github.com/johnabelardom/laravel-awseb-bundle

I deployed successfully but the sqs queue does not execute. I changed the command to command=/usr/bin/php /var/app/current/artisan queue:work sqs but to no avail

@fabioselau077 sorry I haven’t worked with sqs before. Can you maybe try removing the database keyword on the artisan queue:work?

and try adding in the ENV QUEUE_CONNECTION=sqs

With a recent AWS EBS instance, easy_install may not work. In that case, this modified script might help.

New setup.sh

#!/bin/bash

# If you want to have more than one application, and in just one of them to run the supervisor, uncomment the lines below,
# and add the env variable IS_WORKER as true in the EBS application you want the supervisor

#if [ "${IS_WORKER}" != "true" ]; then
#    echo "Not a worker. Set variable IS_WORKER=true to run supervisor on this instance"
#    exit 0
#fi

echo "Supervisor - starting setup"
. /opt/elasticbeanstalk/deployment/env

if [ ! -f /usr/local/bin/supervisord ]; then
    echo "installing supervisor"
    python3 -m ensurepip
    pip3 install supervisor
else
    echo "supervisor already installed"
fi

if [ ! -d /etc/supervisor ]; then
    mkdir /etc/supervisor
    echo "create supervisor directory"
fi

if [ ! -d /etc/supervisor/conf.d ]; then
    mkdir /etc/supervisor/conf.d
    echo "create supervisor configs directory"
fi

. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisor_laravel.conf > /etc/supervisor/conf.d/supervisor_laravel.conf

if ps aux | grep "[/]usr/bin/supervisord"; then
    echo "supervisor is running"
else
    echo "starting supervisor"
    /usr/local/bin/supervisord
fi

/usr/local/bin/supervisorctl reread
/usr/local/bin/supervisorctl update

echo "Supervisor Running!"

Is this working for you at the moment? I get the files and supervisord daemon to start initially, but it exists after a minute (the whole supervisord process)...

does anyone experienced having different SQS URL being use when supervisor process the SQS message? I think that the environment variables does not take effect.

@aliGIThaider @collisiondetection @adityar15 @ashiquepclilac I created a git repo for the sample bundle I used in my project: https://github.com/johnabelardom/laravel-awseb-bundle

I deployed successfully but the sqs queue does not execute. I changed the command to command=/usr/bin/php /var/app/current/artisan queue:work sqs but to no avail

We used :
command=/usr/bin/php /var/www/html/artisan queue:work --sleep=3 --tries=20 ;
`

above all .ebextenstion files have succesfully deployed to beanstalk.But i no log files (supervisor_laravel.err.log, supervisor_laravel.out.log) have created .Is that permission problem ?? if so means how can i add permisssion commands ?

Direct those logs to /var/log/php-fpm/www-error.log they will get sent to cloudwatch, then you can filter them out.

With a recent AWS EBS instance, easy_install may not work. In that case, this modified script might help.

New setup.sh

#!/bin/bash

# If you want to have more than one application, and in just one of them to run the supervisor, uncomment the lines below,
# and add the env variable IS_WORKER as true in the EBS application you want the supervisor

#if [ "${IS_WORKER}" != "true" ]; then
#    echo "Not a worker. Set variable IS_WORKER=true to run supervisor on this instance"
#    exit 0
#fi

echo "Supervisor - starting setup"
. /opt/elasticbeanstalk/deployment/env

if [ ! -f /usr/local/bin/supervisord ]; then
    echo "installing supervisor"
    python3 -m ensurepip
    pip3 install supervisor
else
    echo "supervisor already installed"
fi

if [ ! -d /etc/supervisor ]; then
    mkdir /etc/supervisor
    echo "create supervisor directory"
fi

if [ ! -d /etc/supervisor/conf.d ]; then
    mkdir /etc/supervisor/conf.d
    echo "create supervisor configs directory"
fi

. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisor_laravel.conf > /etc/supervisor/conf.d/supervisor_laravel.conf

if ps aux | grep "[/]usr/bin/supervisord"; then
    echo "supervisor is running"
else
    echo "starting supervisor"
    /usr/local/bin/supervisord
fi

/usr/local/bin/supervisorctl reread
/usr/local/bin/supervisorctl update

echo "Supervisor Running!"

@adityar15 Super thank you! I needed this one since I upgraded my EBS platforms