configuring apcupsd to suspend all running VM and then shutdown the esxi 5.5 u1 host

configure_apcupsd

###############################################################################################################
# these are instructions for automating the suspend and shutdown of esxi vm's and host in case of a
# power failure.
# works with apc smartups 750xl and esxi 5.5u1
###############################################################################################################

0# make a new VM and install Ubuntu-Server on it

1# install apcupsd
  apt-get install apcupsd

2# configure /etc/apcupsd/apcupsd.conf

3# enable apcupsd
  vim /etc/default/apcupsd
    ISCONFIGURED=yes

4# restart apcupsd
  sudo service apcupsd restart
-------------------------------------------------
5# enable ssh on esxi
  configuration-tab - security profile - properties (right upper corner)

6# copy shutdown_esxi.sh on esxi into datastore e.g. "/vmfs/volumes/MyDataStore/scripts

7# make the file executable
  chmod +x shutdown_esxi.sh
--------------------------------------------------
8# configure passwordless ssh access on esxi
  make keys on ubuntu server (as root, because apcupsd runs under root)
    sudo -s
    ssh-keygen -t rsa -b 2048
  copy key to esxi server
    cat /root/.ssh/id.dsa.pub | ssh [email protected] ‘cat >> /etc/ssh/keys-root/authorized_keys’
  now automated login should work on ubuntu server
    ssh [email protected] (the first time it asks to store esxi's key in local database, answer with "yes")
---------------------------------------------------
9# configure apc-script to run remote script on esxi
  copy "doshutdown" to /etc/apcupsd (you mustn't rename it, otherwise apcupsd won't run it automatically)
  make it executable
    chmod +x doshutdown
#10 restart apcupsd
  sudo service apcupsd restart
---------------------------------------------------

doshutdown

#!/bin/sh
WALL=wall

# this script is triggered in case the power fails

echo "suspending all VMs on ESXI..." | ${WALL}

# configured ssh-automatic-access with keys
ssh [email protected] "nohup /vmfs/volumes/MyDataStore/scripts/shutdown_esxi.sh > /dev/null 2>&1 &"

echo "finished shutting ESXI down..." | ${WALL}

# exit code 99 - apccontrol stops after this script, so no shutdown of this host. this is for testing purposes
# exit code 0 - apccontrol continues with shutdown after this script
exit 99

shutdown_esxi.sh

#/bin/sh
 
##########################################################################################################################
#courtesy of http://www.c-note.dk/2011/12/04/wmware-esxi-suspend-all-guests/
#
#copy this file somewhere to datastore
#e.g. ‘/vmfs/volumes/myDataStore/scripts/’
#
# looks which vm's are running, sends them into suspend, waits until suspending-process is finished, then powers off esxi
# completely
##########################################################################################################################
 
VMS=`vim-cmd vmsvc/getallvms | grep -v Vmid | awk '{print $1}'`
for VM in $VMS ; do
  PWR=`vim-cmd vmsvc/power.getstate $VM | grep -v "Retrieved runtime info"`
  if [ "$PWR" == "Powered on" ] ; then
    name=`vim-cmd vmsvc/get.config $VM | grep -i "name =" | awk '{print $3}' | head -1 | cut -d "\"" -f2`
    echo "Powered on: $name"
    echo "Suspending: $name"
    vim-cmd vmsvc/power.suspend $VM > /dev/null &
  fi
done
 
while true ; do
  RUNNING=0
  for VM in $VMS ; do
    PWR=`vim-cmd vmsvc/power.getstate $VM | grep -v "Retrieved runtime info"`
    if [ "$PWR" == "Powered on" ] ; then
      echo "Waiting..."
      RUNNING=1
    fi
  done
  if [ $RUNNING -eq 0 ] ; then
    echo "Gone..."
    break
  fi
  sleep 1
done
 
echo "Now we shutdown the host..."
/sbin/shutdown.sh && /sbin/poweroff

Glad this helps. We have lots of powerfailures here, so I needed this. Thanks for the info! I will try poweroff. At the time i wrote this, poweroff just killed the VM afaik.

And sorry, I don't use xenserver, so I'm not able to help you...

Suspending the VMs and Shutting down the ESXI host works reliably.
However it seems that after ESXI reboot suspended VMs won't restart, even when autostart is activated for these VMs in ESXI.
Is this by design and is there a way to poweroff the VM's instead of suspending?

edit:
i tried replacing
vim-cmd vmsvc/power.suspend $VM > /dev/null &
with
vim-cmd vmsvc/power.shutdown $VM > /dev/null &

but this doesn't seem to work

According to this article from vmware https://kb.vmware.com/s/article/1004340
you can use vmsvc/power.off

for auto-restarting:
I have 6 VMs and two of them are configured for restarting automatically after reboot, but I'm still running version 5.5 on the esxi host. It may be possible that they changed something in newer versions or it may be a bug.

I realized it actually does work. I had not only have to activate restart for each VM but also have to activate it globally in ESXI. Now it does work as expected.

On ESXi 6.0, the script works perfectly, apart from shutting down the host.

It successfully suspends all the VM's, but when the host shuts down, it terminates all the VM's. When it powers back up, the VM's bootup from scratch. Any thoughts?

Are you sure the VM's are successfully suspended? It sounds to me like it starts suspending the VM's but is not waiting to finish and shuts off the host mid suspending. So since the process never finished, the VM will be restarted from scratch. Maybe measure the time it takes to suspend all the machines without shutting down the host and then try the script again and compare the times.
I haven't started my host in over a year and because it is really old, I cannot update it to a newer version of Esxi.

Thanks, yes the script does work correctly. On the esx host side the script runs in a loop, whilst checking the vm status and then exists the script when they have actually suspended. This function works fine when running the script remotely from the apcupsd server using the plink command.......... I may try swapping the command out to ssh with private key sync

…

________________________________ From: gschora ***@***.***> Sent: Monday, 2 January 2023 08:20 To: gschora Cc: Comment Subject: Re: gschora/configure_apcupsd @gschora commented on this gist.

________________________________ Are you sure the VM's are successfully suspended? It sounds to me like it starts suspending the VM's but is not waiting to finish and shuts off the host mid suspending. So since the process never finished, the VM will be restarted from scratch. Maybe measure the time it takes to suspend all the machines without shutting down the host and then try the script again and compare the times. I haven't started my host in over a year and because it is really old, I cannot update it to a newer version of Esxi. - Reply to this email directly, view it on GitHub<https://gist.github.com/a10f0692e6e691aa1af8#gistcomment-4420914> or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAXOXNSMHED5CMPX3GTIKL3WQKFTVBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFQKSXMYLMOVS2I5DSOVS2I3TBNVS3W5DIOJSWCZC7OBQXE5DJMNUXAYLOORPWCY3UNF3GS5DZVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVAYTGMRSG4YDMMFHORZGSZ3HMVZKMY3SMVQXIZI>. You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Actually you may be onto something here...I think there was something with ssh-ing into the host with the key.
Please let me know if it works, just out of curiousity.

Thanks, it is really helpful.
I have another appoach, run apcaccess in ESXI to get UPS status from apcupsd host.

1. Download apcupsd debian package: https://packages.debian.org/search?keywords=apcupsd (Try different version to match libc version of your esxi, bullseye works for my esxi 7)
2. Extract the apcaccess binary file from the package. (Google "how to extract deb file"), I used https://convertio.co/deb-tgz/
3. Save apcaccess to /vmfs/volumes/datastore1/scripts/ of Esxi host.
4. To get UPS status run /vmfs/volumes/datastore1/scripts/apcaccess -h X.X.X.X:3551 which X.X.X.X is the host ip connected to UPS with USB running apcupsd. (You may need to enable wold in firewall rule to allow outgoing)
5. Write a script to poll UPS status and shutdown, for example:
   #!/bin/sh
TLF=`/vmfs/volumes/datastore1/scripts/apcaccess -h 10.0.0.6:3551|grep TIMELEFT |awk '{print $3}'`
TL=${TLF%.*}
#Call shutdown script if time left is less than 10 min
if [ $TL -lt 10 ]
then
echo UPS $TL min left, shutdown in 1 mins
sleep 1m
/vmfs/volumes/datastore1/scripts/shutdown_esxi.sh
#else
#echo $TL min left
fi

6. vi /var/spool/cron/crontabs/root , add the below line * * * * * /vmfs/volumes/datastore1/scripts/pollups.sh > /tmp/pollups.out 2>&1 7. Stop and start crond

kill `cat /var/run/crond.pid`
/usr/lib/vmware/busybox/bin/busybox crond