Domoticz restarter script (or any service)

check_proc.sh

#!/bin/bash

# Author: Przemyslaw W. [saper_2]
# Date: 2019-08-01

# this script check for running process and (re)start it's service. After 4 failures it'll reboot system.
# use "-c" parameter for suppress console output messages if script will be run from cron.
# you can rename this script as you fit (e.g. check_domoticz.sh :P )
# You cat put this in crontab (run script every 5min):
# */5 * * * * /home/pi/check_proc.sh -c
# 
# in /var/log/syslog you'll get info if restart fails.

# to install msmtp, msmtp-mta & mail: sudo apt install msmtp msmtp-mta mailutils

# process name & service name (e.g. domoticz)
PROC_NAME="domoticz"
PROC_NAME_SERVICE="domoticz"
# leave empty if mail and msmtp are not used/installed.
MAIL_TO=""

COUNT_FILE=/tmp/restart_counter_$PROC_NAME

SHELL_RUN=1
# check if script is run from shell or by cron (startup script?)
if [ "$1" == "-c" ]; then
        SHELL_RUN=0
fi
# check if process is running
#pids="$(ps -A|grep $PROC_NAME)"
#echo -e "\e[36mPids result: $pids \e[0m"
# '-q' switch not exists in Debian Stretch (9.0), it was added in Debian Buster (9.9/10.0), so direct out to /dev/null 
#pidof -q $PROC_NAME
pidof $PROC_NAME > /dev/null

#if [ -n $pids ]; then
if [ $? -ne 0 ] ; then
        logger "Application $PROC_NAME is not running, attempting to restart it's service"
        if [ $SHELL_RUN -ne 0 ]; then
                echo -e "\e[31mNot found any process with name '$PROC_NAME'.\e[0m\r\nAttempting to (re)start service..."
        fi
        RESTART_COUNT=1
        # check if count file exists, if so, then read restart attempt count from it
        if [ -a $COUNT_FILE ]; then
                RESTART_COUNT=$(<$COUNT_FILE)
        fi
        # if there were more that 3 restart attmepts already reboot system.
        if [ $RESTART_COUNT -gt 3 ]; then
                if [ $SHELL_RUN -ne 0 ]; then
                        echo -e "\e[91m*** PERFORMING SYSTEM RESTART ***\e[0m"
                fi
                logger "Restarting '$PROC_NAME' service '$PROC_NAME_SERVICE' failed $RESTART_COUNT times. Issusing SYSTEM REBOOT..."
                # mail
                if [ -n "$MAIL_TO" ]; then
                        echo "Unable to restart '$PROC_NAME' after $RESTART_COUNT attempts, performing system reboot." | mail -s "$HOSTNAME - System reboot" $MAIL_TO
                fi
                # perform reboot
                sudo reboot
                #echo -e "\e[31m******************** REBOOT ******************\e[0m" #debug
                # clear file
                echo -n "0" > $COUNT_FILE
                exit 0
        fi
        # incr. restart count
        ((RESTART_COUNT++))
        # save new count back to file
        echo -n "$RESTART_COUNT" > $COUNT_FILE
        # try to restart service
        if [ $SHELL_RUN -ne 0 ]; then
                echo -e "\e[93mTrying to restart $PROC_NAME service '$PROC_NAME_SERVICE'...\e[0m"
        fi

        # restart service
        #pidof $PROC_NAME
        systemctl restart $PROC_NAME_SERVICE
        RESTART_RES=$?

        if [ $RESTART_RES -ne 0 ]; then
                logger "Restarting '$PROC_NAME' service '$PROC_NAME_SERVICE' failed $RESTART_COUNT times."
        fi
        if [ $SHELL_RUN -ne 0 ]; then
                if [ $RESTART_RES -eq 0 ]; then
                        echo -e "\e[92mRestarted.\e[0m"
                else
                        echo -e "\e[91mRestart FAILED.\e[0m"
                fi
        fi
else
        # running, reset counter
        echo -n "0" > $COUNT_FILE
        #rm $COUNT_FILE 2>&1 1>/dev/null

        if [ $SHELL_RUN -ne 0 ]; then
                echo -e "\e[32mProcess $PROC_NAME is running.\e[0m"
        fi

        exit 0
fi

mail-install.md

Here is description how to install mail transfer agent.

1. install mail, msmtp, msmtp-mta: sudo apt install msmtp msmtp-mta mailutils
2. copy-paste and enter your mail settings in file /etc/msmtprc (file below msmtprc , settings defaut for gmail)
3. Set msmtp as default mail-mta agent, edit file: /etc/mail.rc , and add at end of file line set mta=/usr/bin/msmtp
4. check it by running (you'll have to log-out and log-in in ssh/console to reload settings): echo "Test mail" | mail -s "Testing mail" "your.mail@domain.com"

Hint, if you want use gmail mail, create Application Passwords in Account settings -> Security (this is available when 2-Factor Authentication is enabled). If you don't have 2-FA enabled, then enable 'Less secure app access' and in Gmail settings (gear icon above email list -> Settings) enable POP/IMAP forwarding. This should enable access to account that you want send from notifications.

WARNING: Enabling any of those on your primary account might get you hacked (if you pi get hacked, intruder will have your gmail password! or use your account to sending SPAM), so I'd advice crerate a new email only for sending notifications 😄

msmtprc

# default settings
# nothing

# Gmail - default account
account default
host smtp.gmail.com
port 587
tls on
tls_starttls on
from "the-mail-sending-notification@gmail.com"
auth on
user "the-mail-sending-notification@gmail.com"
password "Uthe-mail-sending-notification-password"
logfile ~/.msmtp.log