CODEWITHSUNDEEP
dockerdocker-composedocker-swarmdocker-stack
Daniel Rahamim
Daniel Rahamim

Reputation: 315

Docker stack deploy using overlay network - inconsistent behavior

I am deploying 2 containers (application and SQL) to the same network using a docker-compose.yml file (Swarm stack deploy). Most of the time, the application has no problems talking to the SQL via its host name as a datasource in the connection string.

However, there are times where it simply can't find it. In order to debug it, I have verified that the overlay network is indeed created in each node, and when inspecting the network on each node, I see that the container does belong to this network.

Moreover, when I run docker exec command to enter the application container, I try to send a ping to the SQL container, and the host name does resolves to the correct IP, but still there is no response back.

This is extremely frustrating, as it only occurs from time to time. Any suggestions of how to debug the issue ?

version: '3.2'
services:
  sqlserver:
   image: xxxx:5000/sql_image
   hostname: sqlserver
   deploy: 
     endpoint_mode: dnsrr
   networks:
      devnetwork:
        aliases:
            - sqlserver

  test:
    image: xxxx:5000/test
    deploy: 
     endpoint_mode: dnsrr
    deploy:
     restart_policy:
        condition: none
     resources:
        reservations:
          memory: 2048M
    networks:
      - devnetwork

networks:
    devnetwork:
        driver: overlay

Upvotes: 3

Views: 1384

Answers (1)

ozlevka
ozlevka

Reputation: 2146

Service discovery and DNS problems on load are known bag in swarm mode. We have this problem a lot of times. You can discover open issues here and here.

If you run heavy use network application consider separate your worker and manager nodes. It's will help to manager execute service discovery well.

You can change the service discovery component and use something as Consul or ZooKeeper as part of your stack implementation.

I would consider using some service mesh for data-bind communication between services. Consul can do it for you. You can earn a lot of benefits from this design pattern. Security and encrypted data communication for example.

Upvotes: 1

Related Questions