CODEWITHSUNDEEP
dockerdocker-composecelerydocker-swarm
Greg0ry
Greg0ry

Reputation: 1069

Docker swarm bringing up too many processes

I'm running some celery workers with docker swarm and noticed that docker service ls shows significantly less processes than system tools (like top)

This is what I get when I run docker service ls

docker service ls
ID                  NAME                             MODE                REPLICAS            IMAGE                                                            PORTS
(...)
mdisx2effvfp        stack_celery                     replicated          1/1                 some:url
(...)

This is what I get when I run ps

ps ax | grep celery
 7606 ?        Ss     0:49 /usr/local/bin/python /usr/local/bin/celery -A my_package_name worker --loglevel=WARNING -Q my_queue_name
 7733 ?        S     11:53 /usr/local/bin/python /usr/local/bin/celery -A my_package_name worker --loglevel=WARNING -Q my_queue_name
 7734 ?        Sl     6:02 /usr/local/bin/python /usr/local/bin/celery -A my_package_name worker --loglevel=WARNING -Q my_queue_name
 7735 ?        S      5:52 /usr/local/bin/python /usr/local/bin/celery -A my_package_name worker --loglevel=WARNING -Q my_queue_name

This is what I have within my docker-compose.yaml

cat docker-compose.yaml
(...)
celery:
  image: some:url
  command: celery -A my_package_name worker --loglevel=WARNING -Q my_queue_name
  depends_on:
    - queue # this is my other container with rabbitmq
  deploy:
    restart_policy:
      condition: any
    replicas: 1
    resources:
      limits:
        memory: 1G
  configs:
    - source: celeryconfig.py
      target: /my_package_name/celeryconfig.py
  networks:
    - backend

Am I missing something?

I would guess (based on my configs of course) docker service ls should show the same number of processes as ordinary system tools...

Please help me understand.

--edit(1)--

I can also confirm that when scaling to zero I get no processes at all:

docker service scale stack_celery=0

When I do that then ps will not show celery processes. As soon as I scale back to 1 I can see in ps that there are (again) 4 processes.

--edit(2)--

OK, I would not think celery will automatically spawn processes to accommodate number of CPU's. -c 1 fixes the "problem".

Upvotes: 1

Views: 1186

Answers (2)

Siyu
Siyu

Reputation: 12089

A few concepts here:

service: given by docker service ls, defined in your docker-compose.yml under the keyword services, like "celery", it is a logic group of docker instances.

process: in Docker for linux, you can see the process running in your container directly on your host with ps. (If you bash into your container and run ps ax | grep celery, you should see almost the same thing.)

Celery works by creating workers. The default number is the number of CPU cores. So with celery worker you get 4 worker process.

Upvotes: 2

BMitch
BMitch

Reputation: 263489

Services, containers, and processes running inside of a container are three different things.

A service is used to deploy one or more containers with the same configuration and maintain the target state. The service ls is showing how many replicas to run of the container, not how many processes are running inside of each container.

A container is an isolated environment to run an application. That environment gets a namespace for things like the filesystem, network, and process id's. Note the host can see processes in all namespaces, but inside of the container namespace, you can only see processes belonging to the same namespace.

Your application inside of the container may spawn multiple processes. When pid 1 inside of the container exits, the container will be stopped, so don't start a server in the background and exit the shell that launched it. With celery, it runs multiple workers. There's just one replica of a container, but that container will have multiple pids inside.

See also: http://docs.celeryproject.org/en/latest/userguide/workers.html

Upvotes: 1

Related Questions