Configuring Celery and its progress bar for a Django-EC2(AWS)-Gunicorn-Nginx production environment

Question

I am trying to implement a Celery progress bar to my Django website, which is now on the Web via AWS EC2, Gunicorn, and Nginx, but I simply cannot get the Celery progress bar to work. The bar is supposed to come on after the visitor clicks the submit button, which triggers a process that is run by Celery and feedback on progress is fed to the frontend via Celery Progress Bar. Celery fires up initially, but then flames out if I hit the submit button on my website, and never restarts until I manually restart it.

Below is the report on the Celery failure:

× celery.service - Celery Service
     Loaded: loaded (/etc/systemd/system/celery.service; enabled; preset: enabled)
     Active: failed (Result: timeout) since Mon 2024-12-09 15:14:14 UTC; 6min ago
    Process: 68636 ExecStart=/home/ubuntu/venv/bin/celery -A project worker -l info (code=exited, status=0/SUCCESS)
        CPU: 2min 51.892s

celery[68636]: During handling of the above exception, another exception occurred:
celery[68636]: Traceback (most recent call last):
celery[68636]:   File "/home/ubuntu/venv/lib/python3.12/site-packages/billiard/pool.py", line 1265, in mark_as_worker_lost
celery[68636]:     raise WorkerLostError(
celery[68636]: billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 15 (SIGTERM) Job: 0.
celery[68636]: """
celery[68636]: [2024-12-09 15:14:13,435: WARNING/MainProcess] Restoring 2 unacknowledged message(s)
systemd[1]: celery.service: Failed with result 'timeout'.
systemd[1]: Failed to start celery.service - Celery Service.
systemd[1]: celery.service: Consumed 2min 51.892s CPU time, 817.8M memory peak, 0B memory swap peak.

Here is my celery.service file:

[Unit]
Description=Celery Service
After=network.target

[Service]
Type=forking
User=ubuntu
Group=www-data
EnvironmentFile=/etc/default/celeryd/celery.conf
WorkingDirectory=/home/ubuntu/project
ExecStart=/home/ubuntu/venv/bin/celery -A project worker -l info

[Install]
WantedBy=multi-user.target

My celery.conf file:

# Name of nodes to start
CELERYD_NODES="w1 w2 w3"

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/home/ubuntu/venv/bin/celery"

# App instance to use
CELERY_APP="project.celery:app"

# How to call manage.py
CELERYD_MULTI="multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=2"

# - %n will be replaced with the first part of the nodename.
# - %I will be replaced with the current child process index
#   and is important when using the prefork pool to avoid race conditions.
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_LOG_LEVEL="INFO"

My Gunicorn configuration file:

"""Gunicorn *development* config file"""

# Django WSGI application path in pattern MODULE_NAME:VARIABLE_NAME
wsgi_app = "project.wsgi:application"
# The granularity of Error log outputs
loglevel = "debug"
# The number of worker processes for handling requests
workers = 2
# The socket to bind
bind = "0.0.0.0:8000"
# Restart workers when code changes (development only!)
reload = True
# Write access and error info to /var/log
accesslog = errorlog = "/var/log/gunicorn/dev.log"
# Redirect stdout/stderr to log file
capture_output = True
# PID file so you can easily fetch process ID
pidfile = "/var/run/gunicorn/dev.pid"
# Daemonize the Gunicorn process (detach & enter background)
daemon = True

(I got the details of the Gunicorn file from Real Python.)

And my Nginx configuration file:

server_tokens               off;
access_log                  /var/log/nginx/mysite.access.log;
error_log                   /var/log/nginx/mysite.error.log;

# This configuration will be changed to redirect to HTTPS later
server {
  server_name               .mysite.com;
  listen                    80;
  location / {
    proxy_pass              http://localhost:8000;
    proxy_set_header        Host $host;
  }

  location /static {
    autoindex on;
    alias /var/www/mysite.com/static/;
  }
}

(I also got the Nginx file details from Real Python.)

I have scoured the Net for clues, but simply can't work out what to do. There's a lot of information on the use of Celery in a development environment (which I can already do), but when it comes to a production environment, that is proving really hard to solve, partly because I'm unable to integrate the solutions out there with the configuration files I have.

I am therefore putting my problem out there. Perhaps, someone can advise on how to integrate the solutions out there with the configuration codes I have presented. Celery aside, my website works with the Gunicorn and Nginx files I have from Real Python, so I really don't want to change them. How to get Celery to work with them; that's where I'm stuck.