manage_tasks.py

# -------------------------------- #
# the main advanatages of celery : #
# -------------------------------- #
# queueing, 
# 0. http://python-rq.org/
# queueing is for invoking a function asynchronously

# retries (the most important feature)::
#  0. https://pypi.org/project/backoff/ :: perfect solution
#  1.https://julien.danjou.info/python-tenacity/
#  2.https://pypi.org/project/retrying/
#  3. https://github.com/ionelmc/python-aspectlib/blob/master/src/aspectlib/contrib.py

# multi-threading :: works on top of the Queue concept/logic
# 0.https://stackoverflow.com/questions/2846653/how-to-use-threading-in-python
# 1.https://docs.python.org/2/library/multiprocessing.html#module-multiprocessing :: much better than threading


# full-blown alternative to celery :: https://django-q.readthedocs.io/en/latest/index.html

# use::

# python-crontab
# --------------------#
# documentation links:
# https://pypi.org/project/python-crontab/  
# http://blog.appliedinformaticsinc.com/managing-cron-jobs-with-python-crontab/
# https://github.com/doctormo/python-crontab  

# https://github.com/manugarri/cron-metrics :: has multiple dependencies
# --------------------- #


# django-integrated solutions
# --------------------------- #
# https://github.com/kraiz/django-crontab  :: no monitoring of cron job state
# https://github.com/Tivix/django-cron :: does monitoring, no queing of jobs
# https://github.com/jgorset/django-kronos
# https://bitbucket.org/wnielson/django-chronograph/

# monitor cron jobs
# ----------------- #
# https://github.com/jamesrwhite/minicron
# https://pythonhosted.org/django-cron-monitor/ :: has webUI for task states
# <https://pypi.org/project/django-cron-monitor/> :: pip install django-cron-monitor
# syslog: https://www.debuntu.org/how-to-remote-syslog-logging-on-debian-and-ubuntu/
# cron's arg :: MAILTO=someuser

# see this thread :: https://stackoverflow.com/questions/573618/django-set-up-a-scheduled-job

# or supervisor
command=/path/to/my_command.py
directory=/home/mwaiharyan/trade_bot/fx
autostart=true
startretries=5
stderr_logfile=/var/log/task_manager/err.log
user=www-data

cron and virtualenv config:
https://stackoverflow.com/questions/3287038/cron-and-virtualenv?rq=1

Create a shell script every-5-seconds.sh using bash while loop as shown below.
$ cat every-5-seconds.sh #!/bin/bash while true do /home/ramesh/backup.sh sleep 5 done

Now, execute this shell script in the background using nohup as shown below. This will keep executing the script even after you logout from your session. This will execute your backup.sh shell script every 5 seconds.

$ nohup ./every-5-seconds.sh &

source https://www.thegeekstuff.com/2011/07/cron-every-5-minutes/

souc

executing cron jobs on specific days of the month,e.g,every last Monday, see http://ashok-linux-tips.blogspot.co.ke/2010/12/executing-cronjobs-on-specific-days-in.html

use 'Monit' to monitor a cron job https://mmonit.com/wiki/Monit/ConfigurationExamples#cron

check process myprocessname matching "myprocessname" start program = "nohup /usr/bin/python /path/testing.py > /tmp/test.out &" stop program = "/usr/bin/killall myprocessname"

Weird :
use a cronjob to run 'Huey' every second,every business day during the trading hours

why celery and not cron ? A reddit thread https://www.reddit.com/r/Python/comments/m2dg8/explain_like_im_five_why_or_why_not_would_celery/