Run laravel queue worker in docker

Question

I want to run php artisan queue:work --daemon command in docker file or docker-compose.yml but if I use command : xxx in docker compose file, ngin x returns 502.
I try to use bash file and again it's not working.
Can anyone help me?
I need to finish this today and really need help.
Docker compose file

version: '3'
services:

  #Nginx Service
  webserver:
    image: nginx:alpine
    container_name: LibraryWebserver
    restart: unless-stopped
    tty: true
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./:/var/www
      - ./nginx/conf.d/:/etc/nginx/conf.d/
    networks:
      - app-network

  #MySQL Service
  db:
    image: mysql:5.7.22
    container_name: Librarydb
    restart: unless-stopped
    tty: true
    ports:
      - "3306:3306"
    environment:
      MYSQL_DATABASE: library
      MYSQL_ROOT_PASSWORD: Library!23
      MYSQL_USER: root
      MYSQL_PASSWORD: Library!23
      SERVICE_TAGS: dev
      SERVICE_NAME: mysql
    networks:
      - app-network
    volumes:
      - dbdata:/var/lib/mysql

  #PHP Service
  app:
    build:
      context: .
      dockerfile: Dockerfile
    image: hoseinnormohamadi/lumen:Library
    container_name: Library
    restart: unless-stopped
    tty: true
    environment:
      SERVICE_NAME: app
      SERVICE_TAGS: dev
    working_dir: /var/www
    volumes:
      - ./:/var/www
    networks:
      - app-network
#Docker Networks
networks:
  app-network:
    driver: bridge
#Volumes
volumes:
  dbdata:
    driver: local

Answer 1

Not sure if this helps but I had the same challenge around a year ago and I ended up creating a separate "worker" container + an additional container for redis which was used as a queue driver. I haven't touched laravel for quite a while and it was one of my first attempts of using docker so you might need to clean things up and adjust to your needs.

For the worker I used the following (service within the docker compose):

  worker:
    build: ./docker-images/worker
    volumes:
    - ./app:/var/www/laravel
    #- ./data/app/storage:/var/www/laravel/storage/
    #- ./docker-images/php-fpm/php71.ini:/usr/local/etc/php/php.ini
    - ./docker-images/worker/supervisor:/etc/supervisor
    networks:
    - backend

This is the content of the Dockerfile for worker (Will post it as is and you can pick which parts of it you might need. The main thing is that your app should run there + you need supervisord which will run as a daemon and restart your queue all the time). Might be quite outdated but should give an idea:

#
#--------------------------------------------------------------------------
# Image Setup
#--------------------------------------------------------------------------
#

FROM php:7.1-fpm

#
#--------------------------------------------------------------------------
# Software's Installation
#--------------------------------------------------------------------------
#
# Installing tools and PHP extentions using "apt", "docker-php", "pecl",
#

RUN apt-get update \
  && apt-get install -y --no-install-recommends \
    curl \
    vim \
    git \
    curl \
    python3.4 \
    python3-pip \
    libmemcached-dev \
    libz-dev \
    libpq-dev \
    libjpeg-dev \
    libpng-dev \
    libfreetype6-dev \
    libssl-dev \
    libmcrypt-dev \
  && rm -rf /var/lib/apt/lists/*

# Install PHP extensions

RUN docker-php-ext-install mcrypt \
  # Install the PHP pdo_mysql extention
  && docker-php-ext-install pdo_mysql \
  # Install the PHP pdo_pgsql extention
  && docker-php-ext-install pdo_pgsql \
  && docker-php-ext-install pcntl sockets bcmath mbstring opcache mysqli gettext \
  # Install the PHP gd library
  && docker-php-ext-configure gd \
    --enable-gd-native-ttf \
    --with-jpeg-dir=/usr/lib \
    --with-freetype-dir=/usr/include/freetype2 && \
    docker-php-ext-install gd


#####################################
# Human Language and Character Encoding Support:
#####################################
RUN apt-get update -yqq && \
    apt-get install -y zlib1g-dev libicu-dev g++ && \
    docker-php-ext-configure intl && \
    docker-php-ext-install intl 

#####################################
# Mongo
#####################################

RUN apt-get install libcurl4-openssl-dev pkg-config -y && \
  apt-get install pkg-config -y && \ 
  apt-get install openssl -y &&  \
  pecl install mongodb && \ 
  docker-php-ext-enable mongodb


####################################
# Composer
####################################
RUN curl -o /tmp/composer-setup.php https://getcomposer.org/installer \
  && curl -o /tmp/composer-setup.sig https://composer.github.io/installer.sig \
  # Make sure we're installing what we think we're installing!
  && php -r "if (hash('SHA384', file_get_contents('/tmp/composer-setup.php')) !== trim(file_get_contents('/tmp/composer-setup.sig'))) { unlink('/tmp/composer-setup.php'); echo 'Invalid installer' . PHP_EOL; exit(1); }" \
  && php /tmp/composer-setup.php --no-ansi --install-dir=/usr/local/bin --filename=composer --snapshot \
  && rm -f /tmp/composer-setup.*


## Source the bash
RUN . ~/.bashrc
#RUN composer global install


#####################################
# Image optimizers:
#####################################
#USER root
#RUN apt-get update -yqq && \
#    apt-get install -y --force-yes jpegoptim optipng pngquant gifsicle \


#####################################
# ImageMagick:
#####################################
USER root
RUN apt-get update -y && \
    apt-get install -y libmagickwand-dev imagemagick && \ 
    pecl install imagick && \
    docker-php-ext-enable imagick 



#####################################
# Exif:
#####################################
RUN docker-php-ext-install exif 

#####################################
# ZipArchive:
#####################################
RUN docker-php-ext-install zip 


#####################################
# PHP REDIS EXTENSION FOR PHP 7.0
#####################################
RUN printf "\n" | pecl install -o -f redis \
    &&  rm -rf /tmp/pear \
    &&  docker-php-ext-enable redis 

#####################################
# pgsql client
#####################################
#RUN apt-get update -yqq && \
#    apt-get install -y postgresql-client

#####################################
# pgsql
#####################################
#RUN apt-get update -yqq && \
#    docker-php-ext-install pgsql

#####################################
# Mysql stuff
#####################################
RUN apt-get update -yqq && \
    apt-get install mysql-client -y


# Install supervisor

RUN apt-get update \
  && apt-get install -y --no-install-recommends \
    supervisor \
  && rm -rf /var/lib/apt/lists/*


ENTRYPOINT ["/usr/bin/supervisord", "-n", "-c",  "/etc/supervisor/supervisord.conf"]

WORKDIR /etc/supervisor/conf.d/

I was using Laravel Horizon. Using it without horizon works as well but needs a slightly different configuration.

As you saw in the docker-compose service declaration, we are mapping local folder with configs. ./docker-images/worker/supervisor:/etc/supervisor.

This folder structure looks like this: docker-images (folder)

• worker (a folder that contains things that belong to that service)
• Dockerfile (contents posted above)
• supervisor (folder that contains contents configs for supervisor)
  • supervisor.conf - configuration file (will post the contents below)
  • conf.d (folder with supervisor's worker config files)
    • horizon.conf (configuration file for laravel horizon but can also be running queue directly instead of running horizon. Will post the contents below)

supervisor.conf sample contents

[unix_http_server]
file=/var/run/supervisor.sock   ; (the path to the socket file)
chmod=0760                       ; sockef file mode (default 0700)
chown=root

[supervisord]
logfile=/etc/supervisor/logs/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
;childlogdir=/etc/supervisor/logs/supervisord/childlog  ; ('AUTO' child log dir, default $TEMP)

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL  for a unix socket

; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files *cannot*
; include files themselves.

[include]
files = /etc/supervisor/conf.d/*.conf

Horizon.conf contents

[program:horizon]
process_name=%(program_name)s
command=php /var/www/laravel/artisan horizon
autostart=true
autorestart=true
user=www-data
redirect_stderr=true
stderr_logfile=/etc/supervisor/logs/horizon/err.log
stdout_logfile=/etc/supervisor/logs/horizon/out.log

For running and stoping supervisor you would need to log into the worker container via $ docker-compose exec worker bash and here are some useful commands:

Check status

$ supervisorctl status

$ supervisorctl

stop job-name // stops the job

stop parser-worker:* // stops the whole group of processes

$ supervisorctl reread // rereads the config file, if you update something in it

$ supervisorctl update // applies all the changes

Alternatively for restarting supervisor or re-reading config files you could just restart worker container.

Other than that, you might also want to read some official supervisor docs. I remember took me quite a few days to set things up so hopefully, this helps you.

Answer 2

Docker file

FROM php:7.4-fpm-alpine

RUN docker-php-ext-install pdo pdo_mysql

RUN apk update && apk add --no-cache supervisor

Docker compose file

version: '3'

networks:
  laravel:

services:

  supervisor:
    build:
      context: .
      dockerfile: supervisor.dockerfile
    volumes:
      - .:/var/www/html:delegated
    networks:
      - laravel
    command: 
      - /bin/sh
      - -c
      - supervisord -c /var/www/html/docker/supervisor/supervisord.conf 
      - supervisorctl  start all

and supervisor conf file

[unix_http_server]
file=/tmp/supervisor.sock   ; the path to the socket file

[supervisord]
logfile=/var/www/html/docker/supervisor/logs/supervisord.log ; main log file; default $CWD/supervisord.log
logfile_maxbytes=5MB        ; max main logfile bytes b4 rotation; default 50MB
logfile_backups=10           ; # of main logfile backups; 0 means none, default 10
loglevel=info                ; log level; default info; others: debug,warn,trace
pidfile=/tmp/supervisord.pid ; supervisord pidfile; default supervisord.pid
nodaemon=false               ; start in foreground if true; default false
minfds=1024                  ; min. avail startup file descriptors; default 1024
minprocs=200                 ; min. avail process descriptors;default 200

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
numprocs=1
redirect_stderr=true
stdout_logfile=/var/www/html/docker/supervisor/logs/worker.log
stopwaitsecs=3600
stdout_logfile_maxbytes=5MB 

will works fine