Celery task (process) is not closed

Question

I am running the following infrastructure: Celery starts Selenium-based browsers in separate tasks

I noticed (with help of htop )that I have tens of unclosed celery processes with firefox-esr and /usr/local/bin/python3.9 /usr/local/bin/celery -A core_app worker ...

How to properly close celery tasks?

My infrastructure in docker-compose.yaml:

version: '3.3'

services:
  django:
    image: harbor.mgns-tech.ru/content-protect/main-app:latest
    command: sh -c "
      sleep 3 && gunicorn --bind 0.0.0.0:8000 core_app.wsgi"
    volumes:
      - ./project/static:/project/static
      - ./project/media:/project/media
      - ./project/logs:/project/logs
    expose:
      - 8000
    env_file:
      - ./.env
    depends_on:
      - db


  db:
    image: postgres:13-alpine
    volumes:
      - pg_data:/var/lib/postgresql/data/
    expose:
      - 5432
    ports:
      - "5433:5432"
    env_file:
      - .env
    environment:
      - POSTGRES_USER=${DB_USER}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=${DB_NAME}

  nginx:
    image: nginx:1.24.0-alpine
    depends_on:
      - django
    env_file:
      - ./.env
    ports:
      - "80:80"
    volumes:
      - ./project/static:/var/www/html/static
      - ./project/media:/var/www/html/media
      - ./project/nginx-conf.d/:/etc/nginx/conf.d

  celery:
    image: harbor.mgns-tech.ru/content-protect/main-app:latest
    command: celery -A core_app worker  --loglevel=info --concurrency=15 --max-memory-per-child=1000000
    volumes:
      - ./project:/usr/src/app
      - ./project/media:/project/media
      - ./project/logs:/project/logs
    env_file:
      - .env
    environment:
    # environment variables declared in the environment section override env_file
      - DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]
      - CELERY_BROKER=redis://redis:6379/0
      - CELERY_BACKEND=redis://redis:6379/0
    depends_on:
      - django
      - redis


  celery-flower:
    image: harbor.mgns-tech.ru/content-protect/main-app:latest
    restart: always
    command: celery -A core_app flower --loglevel=INFO --port=9090
    ports:
      - "9090:9090"
    env_file:
      - .env
    depends_on:
      - django
      - redis


  redis:
    image: redis:5-alpine
    volumes:
      - ./project:/usr/src/app
      - ./project/media:/project/media
      - ./project/logs:/project/logs
      - ./redis-data:/data
    ports:
     - "6379:6379"
    sysctls:
      - net.core.somaxconn=65535

Example of celery task

@shared_task()
def check_urls(parsing_result_ids: List[int]):
    """
    Run Selenium-based yandex parser
    the parser exctracts data and saves in database
    """
    try:
        logger.info(f"{datetime.now()} Start check_urls")
        parser = YandexParser() # Selenium object
        parsing_results = ParsingResult.objects.filter(pk__in=parsing_result_ids).exclude(status="DONE")
        logger.info(f"{datetime.now()} Start check_urls: {len(parsing_results)} urls")

        parser.check_parsing_result(parsing_results)

        logger.info(f"{datetime.now()} Stop check_urls")
    except Exception as e:
        full_trace = traceback.format_exc()
        logger.info(f"Exception in check urls task: {full_trace}")
    finally:
        if 'parser' in locals():
            parser.stop() # stops selenium