How to setup my crontab in order to execute process_tasks in Django?

Question

I decided to use a library for my Django project called django-background-tasks (link to the documentation: https://django-background-tasks.readthedocs.io/en/latest/). I want to deploy my Django application to a Linux server (Ubuntu 19.0.4). How should I write the crontab in order to call the command "process_tasks" every five seconds?

Here Running a cron every 30 seconds is a workaround to achieve the seconds part, but since I am new to this part of the job (deploying and automation of process), how should I create my crontab file in order to achieve my desired purpose?

I'll be using process_tasks for a lot of different functionalities like: do some analysis at night and send results in the morning, expire some codes, etc. So basically I'll need to be running it almost constantly.

Thank you in advance for any suggestion, if you need something more I would be happy to provide it to you.

Answer 1

Since you know cron only allows for a minimum of one minute but you want to run it every 5 seconds.

how about writing a shell script (running as a service via supervisor) that runs your task in an infinite loop. This script will sleeps after every run for 5 seconds.

The only difference between this service and a cron is: A cron will fork a process every time it runs, a process runs your job and periodically check if the job is finished in order to clean it up, and this script (as service) will not work like cron but it'll do the job I believe.

#!/bin/bash
while true; do
    # your code here, 
    # call your python file which initializes 
    # django variables (or whatever you want) and do the needful.
    sleep 5;
done

You can configure a file process_django_tasks.sh(this file contains above code) with supervisor as a service so it runs on boot time and you have a control to start and stop with quick commands.

To test quickly you can easily run sh process_django_tasks.sh

If you could tell us why exactly you want your script to run every 5 seconds. Maybe we all suggest a better way than running a script every 5 sec.

Answer 2

I'd recommend using Supervisor (http://supervisord.org) to run python manage.py process_tasks, which will then monitor for the tasks you scheduled in your code and run them based on their repeat syntax:

To run your task every 5 seconds:

function_to_call(var, repeat=5, repeat_until=None)

I'd also recommend Supervisor to run your entire Django project.

Looking at the documentation the process_tasks command by default runs every 5 seconds checking for new tasks to run. Not sure if you just wanted to check every 5 seconds or actually run a specific task every 5 seconds.

Answer 3

Not knowing any details, this may be a completely inapplicable alternative. However, if there is a guarantee that there will be nothing to be done until (an) instance(s) of (a) particular model(s) is saved, and provided that this save doesn't happen far too frequently, then you might look at Django post_save signals instead. Whenever that save happens, either execute the task, or execute the task if it wasn't run already in the last five seconds.