Skip to content

Instantly share code, notes, and snippets.

@ilyasst
Last active April 15, 2024 04:34
Show Gist options
  • Save ilyasst/28bee4ee7ebb89e01284bc7e048346e9 to your computer and use it in GitHub Desktop.
Save ilyasst/28bee4ee7ebb89e01284bc7e048346e9 to your computer and use it in GitHub Desktop.
Run a python script forever using systemd

Run a python script forever

In this section, we are going to show how to run a python script as a systemd service, allowing you to boot it at the start of a Linux machine and to maintain it alive.

Test method: Telegram Bot

We are going to use a very basic Telegram bot in order to test that our script will:

  1. Automatically start when the machine boot
  2. Automatically restart when it crashes/exits for whichever reason

If the bot is alive, then it means that our method works. Of course, we will also be able to check the status of the service through systemd, but just to be sure ... This bot is going to send us

  1. Create a ~/Temp folder on your Raspberry Pi using SSh
  2. Create a virtual environment in ~/Temp, you should already have venv we installed it at the beginning.
dietpi@solidsnake:~/Temp$ python3 -m venv .env
  1. Load the virtual environment: source .env/bin/activate

  2. Let's also update pip, and install a pip we need.

(.env) dietpi@solidsnake:~/Temp$ pip install --upgrade pip
(.env) dietpi@solidsnake:~/Temp$ pip install pytelegrambotAPI
  1. Create a bot.py: nano bot.py:

Let's give our bot:

  • A help message
# -*- coding: utf-8 -*-
import telebot

bot = telebot.TeleBot("725494896:AAFyUSf5e_QyYve0lmtr0AN7EcKvgy_aB-U")

@bot.message_handler(commands=['help', 'aide'])
def send_welcome(message):
    text = """
Help !
"""
    bot.reply_to(message, text)
    
bot.polling()
  1. Test your script: python bot.py

Let's now make this script into a systemd service.

On Dietpi

Important

All the paths in your scripts have to be absolute paths, there can be no relative path in your scripts. If there are relative paths that you must keep, you will have to change your current working directory by retrieving

  1. Modify the python script, add first line: #!/home/dietpi/Temp/.env/bin/python3 which is the path to the python in the virtual env
  2. chmod +x the python file to make it executable, it will execute with the python you specified (no need to load the venv yourself, just execute without specifying the python bin)
  3. Type the following command in your terminal to add a systemd service sudo systemctl edit --force --full dummy.service:
[Unit]
Description=Dummy Service
Wants=network.target
After=network.target

[Service]
ExecStartPre=/bin/sleep 10
ExecStart=/home/dietpi/Temp/bot.py
Restart=always

[Install]
WantedBy=multi-user.target
  • Note that ExecStart is the path to the Python file directly if you made it an executable using the right virtual environment. If you did not, then you have to specify a python binary to execute it.

  • We have to add an ExecStartPre delay otherwise the service keeps tries to start before internet is even available and we get this error:

● dummy.service
   Loaded: loaded (/etc/systemd/system/dummy.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2019-10-29 00:26:18 GMT; 5min ago
  Process: 442 ExecStart=/home/dietpi/guerab/guerab.py (code=exited, status=1/FAILURE)
 Main PID: 442 (code=exited, status=1/FAILURE)

Oct 29 00:26:18 solidsnake guerab.py[442]:     timeout=(connect_timeout, read_timeout), proxies=proxy)
Oct 29 00:26:18 solidsnake guerab.py[442]:   File "/home/dietpi/guerab/.env/local/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
Oct 29 00:26:18 solidsnake guerab.py[442]:     resp = self.send(prep, **send_kwargs)
Oct 29 00:26:18 solidsnake guerab.py[442]:   File "/home/dietpi/guerab/.env/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
Oct 29 00:26:18 solidsnake guerab.py[442]:     r = adapter.send(request, **kwargs)
Oct 29 00:26:18 solidsnake guerab.py[442]:   File "/home/dietpi/guerab/.env/local/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
Oct 29 00:26:18 solidsnake guerab.py[442]:     raise ConnectionError(err, request=request)
Oct 29 00:26:18 solidsnake guerab.py[442]: requests.exceptions.ConnectionError: ('Connection aborted.', gaierror(-3, 'Temporary failure in name resolution'))
Oct 29 00:26:18 solidsnake systemd[1]: dummy.service: Main process exited, code=exited, status=1/FAILURE
Oct 29 00:26:18 solidsnake systemd[1]: dummy.service: Failed with result 'exit-code'.
  • We also add a Restart flag in order to get systemd to always restart the script if it were to ever fail

Use Ctrl X + Y to save and exit when you finished editing.

  1. Enable the service sudo systemctl enable dummy.service
  2. Reboot, wait for 30 seconds
  3. Try to contact your bot with /help, All good !
  4. SSH into your RPi
  5. Check your service status: sudo systemctl status dummy.service
  6. Manipulate your service:
sudo systemctl stop dummy.service          #To stop running service 
sudo systemctl start dummy.service         #To start running service 
sudo systemctl restart dummy.service       #To restart running service 
  1. Let's validate that it will really restart on crash. Let's add a function to our bot that simply kills the script. By killing, I mean that we are going to create an error in order to get the script to crash. When we are working on a telegram bot script, each function is kind of loaded separately, we are going to create an error in a new function and use it to check if the bot truely restarts, add this:
#!/home/dietpi/Temp/.env/bin/python3
# -*- coding: utf-8 -*-
import telebot

bot = telebot.TeleBot("TELEGRAM_BOT_TOKEN")

@bot.message_handler(commands=['help', 'aide'])
def send_welcome(message):
    text = """
Help !
"""
    bot.reply_to(message, text)

@bot.message_handler(commands=["kill"])
def get_kill(message):
    print(a)

bot.polling()

If you try ths script (in your Virtual Environment not as a service), you will see that the script will return the /help command, but it will simply crash if you try to run the /kill command which tries to print a variable a that was never defined. Because python sees each telegram bot function as a separate function, it does not check that all variables exist before, as a variable can be defined with an incoming Telegram message.

  1. Deploy (copy/paste) this new script on the Raberry pi, then reboot the RPi, so it properly loads as the new systemd service
  2. Wait for 30 seconds, then contact your bot using /help to check that it is online
  3. Use the kill command, the bot should die, wait for a bit more than 10 seconds, as we have a 10 second timer set prior to starting the script
  4. Try /help again, IT WORKS!

In order to delete the service:

sudo systemctl disable dummy.service

then reboot.

Freqtrade on Dietpi


UPDATE (March 2020)

The freqtrade code was updated in February 2020 in such a way that freqtrade should now be run from the freqtrade/freqtrade/main.py file instead of freqtrade/bin/freqtrade. This means that there is a difference between the file being edited in the video and the one edited in this document. The version presented in this document should be used.

In addition, the DefaultStrategy was removed from freqtrade. You will thus get an error if you try to run it, use BBRSI instead.

Cheers !


  1. Go to ~/freqtrade/freqtrade, and edit the file called main.py using nano:
nano main.py
  1. Change the first line so that your new file looks like this:
#!/home/dietpi/freqtrade/.env/bin/python3

"""
Main Freqtrade bot script.
Read the documentation to know what cli arguments you need.
"""

from freqtrade.exceptions import FreqtradeException, OperationalException
import sys
# check min. python version
if sys.version_info < (3, 6):
    sys.exit("Freqtrade requires Python version >= 3.6")

# flake8: noqa E402
import logging
from typing import Any, List

from freqtrade.commands import Arguments


logger = logging.getLogger('freqtrade')


def main(sysargv: List[str] = None) -> None:
    """
    This function will initiate the bot and start the trading loop.
    :return: None
    """

...
  1. Add the freqtrade systemd service using:
sudo systemctl edit --force --full freqtrade.service
  1. Paste the following:
[Unit]
Description=Freqtrade Service
Wants=network.target
After=network.target

[Service]
ExecStartPre=/bin/sleep 10
ExecStart=/home/dietpi/freqtrade/freqtrade/main.py trade -c /home/dietpi/freqtrade/config.json --strategy DefaultStrategy --userdir /home/dietpi/freqtrade/
Restart=always

[Install]
WantedBy=multi-user.target
  1. Use Ctrl+X then Y followed by Enter to save the new serviced.

  2. Enable the service sudo systemctl enable freqtrade.service

  3. Reboot the Raspberry pi: sudo reboot now

Freqtrade on VM


UPDATE (March 2020)

The freqtrade code was updated in February 2020 in such a way that freqtrade should now be run from the freqtrade/freqtrade/main.py file instead of freqtrade/bin/freqtrade. This means that there is a difference between the file being edited in the video and the one edited in this document. The version presented in this document should be used.

In addition, the DefaultStrategy was removed from freqtrade. You will thus get an error if you try to run it, use BBRSI instead.

Cheers !


  1. Go to ~/freqtrade/freqtrade, and edit the file called main.py using nano:
nano main.py
  1. Change the first line so that your new file looks like this:
#!/root/freqtrade/.env/bin/python3

"""
Main Freqtrade bot script.
Read the documentation to know what cli arguments you need.
"""

from freqtrade.exceptions import FreqtradeException, OperationalException
import sys
# check min. python version
if sys.version_info < (3, 6):
    sys.exit("Freqtrade requires Python version >= 3.6")

# flake8: noqa E402
import logging
from typing import Any, List

from freqtrade.commands import Arguments


logger = logging.getLogger('freqtrade')


def main(sysargv: List[str] = None) -> None:
    """
    This function will initiate the bot and start the trading loop.
    :return: None
    """

...
  1. Add the freqtrade systemd service using:
sudo systemctl edit --force --full freqtrade.service
  1. Paste the following:
[Unit]
Description=Freqtrade Service
Wants=network.target
After=network.target

[Service]
ExecStartPre=/bin/sleep 10
ExecStart=/root/freqtrade/freqtrade/main.py trade -c /root/freqtrade/config.json --strategy DefaultStrategy --userdir /root/freqtrade/
Restart=always

[Install]
WantedBy=multi-user.target
  1. Use Ctrl+X then Y followed by Enter to save the new serviced.

  2. Enable the service sudo systemctl enable freqtrade.service

  3. Reboot the Raspberry pi: sudo reboot now

Additional resources
  1. https://raspberrypi.stackexchange.com/questions/90148/running-a-python-telegram-bot-at-startup-and-24-7
  2. https://www.digitalocean.com/community/tutorials/how-to-configure-a-linux-service-to-start-automatically-after-a-crash-or-reboot-part-2-reference
  3. Exit Telegram Bot: python-telegram-bot/python-telegram-bot#801
@ebuka-odih
Copy link

Please, i was using the service file like you explained on Udemy, so it was working for two days and i went to change some settings on my config file, after doing so it has refused to work again.
image

@Timmeeehhh
Copy link

I had problems running the code, because it couldn't find the strategy. Adding the location helped me out.
For anyone running into the same problem, copy the text below

[Unit]
Description=Freqtrade Service
Wants=network.target
After=network.target

[Service]
ExecStartPre=/bin/sleep 10
ExecStart=/root/freqtrade/freqtrade/main.py trade -c /root/freqtrade/config.json --strategy DefaultStrategy --strategy-path /root/freqtrade/user_data/strategies --userdir /root/freqtrade/
Restart=always

[Install]
WantedBy=multi-user.target

@tutecode
Copy link

What is that?

Created symlink /etc/systemd/system/multi-user.target.wants/freqtrade.service → /etc/systemd/system/freqtrade.service.

I cannot run the script

@mohirad
Copy link

mohirad commented Nov 5, 2020

I tried to see if this script for systemd service doesn't have any problems in Google cloud VM instance so I can proceed with the next steps:
mohirad@instance-1:~/freqtrade/freqtrade$ /root/freqtrade/freqtrade/main.py trade -c /root/freqtrade/config.json --strategy DefaultStrategy --userdir /root/freqtrade/

But I got this error:
-bash: /root/freqtrade/freqtrade/main.py: Permission denied

Can you please help me out with this?

@ebuka-odih
Copy link

ebuka-odih commented Nov 5, 2020 via email

@Novitec1
Copy link

Novitec1 commented Mar 9, 2021

Are there any updates to this? I have tried this exactly as it was written with no luck. Any help would be appreciated. Thank you.

@AhmedSakrr
Copy link

AhmedSakrr commented Oct 27, 2021

What’s that?

Created symlink /etc/systemd/system/multi-user.target.wants/freqtrade.service → /etc/systemd/system/freqtrade.service.

Active: Inactive (dead)

  • why the Service is not working? I’m trying it with the latest freqtrade develop branch version

@Johan1974
Copy link

When i try to run the command i get the following:

root@localhost:/freqtrade/.env/bin# /root/freqtrade/freqtrade/main.py trade -c /root/freqtrade/config.json --strategy bbrsi --userdir /root/freqtrade/ Traceback (most recent call last): File "/root/freqtrade/freqtrade/main.py", line 10, in <module> from freqtrade.util.gc_setup import gc_set_threshold ModuleNotFoundError: No module named 'freqtrade' root@localhost:/freqtrade/.env/bin#

@amjad-12
Copy link

@ilyasst hello mr.ilyass please can you update the ExecStart because it don't work ever

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment