NATS Client throws NATSConnectionException timeout

Question

I'm trying to connect to NATS that uses docker compose.CreateConnection throws a NATSConnectionException: timeout exception.

By the way, I took the docker compose from the official documentation.

How do I fix it?

try
{
    _connection = new ConnectionFactory().CreateConnection("nats://nats:4222");
}
catch (Exception ex)
{
    Console.WriteLine(ex.Message);
}

version: '3.9'

services:
  consul-server:
    image: hashicorp/consul:1.13.1
    container_name: consul-server
    restart: always
    volumes:
      - ./consul/server.json:/consul/config/server.json:ro
    networks:
      - hashicorp
    ports:
      - "8500:8500"
      - "8600:8600/tcp"
      - "8600:8600/udp"
    command: "agent -bootstrap-expect=1"

  nats:
    image: nats
    ports:
      - "8222:8222"
    command: "--cluster_name NATS --cluster nats://0.0.0.0:6222 --http_port 8222 "
    networks:
      - nats

  nats-1:
    image: nats
    command: "--cluster_name NATS --cluster nats://0.0.0.0:6222 --routes=nats://ruser:T0pS3cr3t@nats:6222"
    networks:
      - nats
    depends_on:
      - nats

  nats-2:
    image: nats
    command: "--cluster_name NATS --cluster nats://0.0.0.0:6222 --routes=nats://ruser:T0pS3cr3t@nats:6222"
    networks:
      - nats
    depends_on:
      - nats

networks:
  hashicorp:
    driver: bridge
  nats:
    name: nats

Edit

I fixed it by using the following docker compose and changing the NATS url to nats://localhost:14222. Mind telling me what was wrong with the previous one because I literally took it from the docs.

version: '3.9'

services:
  consul-server:
    image: hashicorp/consul:1.13.1
    container_name: consul-server
    restart: always
    volumes:
      - ./consul/server.json:/consul/config/server.json:ro
    networks:
      - hashicorp
    ports:
      - "8500:8500"
      - "8600:8600/tcp"
      - "8600:8600/udp"
    command: "agent -bootstrap-expect=1"

  nats-1:
    command:
      - "--debug"
      - "--cluster"
      - "nats://0.0.0.0:6222"
      - "--http_port"
      - "8222"
      - "--port"
      - "4222"
    image: "nats:2.1.4"
    networks:
      - main
    ports:
      - "14222:4222"
      - "18222:8222"
  nats-2:
    command:
      - "--debug"
      - "--cluster"
      - "nats://0.0.0.0:6222"
      - "--http_port"
      - "8222"
      - "--port"
      - "4222"
      - "--routes"
      - "nats://nats-1:6222"
    image: "nats:2.1.4"
    networks:
      - main
    ports:
      - "24222:4222"
      - "28222:8222"
  nats-3:
    command:
      - "--debug"
      - "--cluster"
      - "nats://0.0.0.0:6222"
      - "--http_port"
      - "8222"
      - "--port"
      - "4222"
      - "--routes"
      - "nats://nats-1:6222"
    image: "nats:2.1.4"
    networks:
      - main
    ports:
      - "34222:4222"
      - "38222:8222"

networks:
  hashicorp:
    driver: bridge
  main:
    driver: bridge
    ipam:
      config:
        - subnet: 172.25.255.0/24
      driver: default

Answer 1

In our case we found that the problem is with multiple connection requests happening simultaneously to the server due to parallelism in our software, it means that we had different components trying to create a connection to server at the same time, although we didn't find in nats document site the exact number of connection requests that the server can handle in the default timeout frame ( 2 seconds ) we created a test that shows the 15 connection requests in parallel will cause the NATSConnectionException: timeout exception. So for that we had 2 solutions:

1. Increase the the time out:

   var defaultOptions = ConnectionFactory.GetDefaultOptions();
   defaultOptions.Timeout = 20000;  
   connection = 
   new ConnectionFactory().CreateEncodedConnection(defaultOptions);
   

2. Wrap the creation of connections with a components that managed the number of connections that you could do concurrently, something like this:

   public class NatsConnectionManager
   {
   
       private readonly SemaphoreSlim _serviceLock = new SemaphoreSlim(1, 5); // number of connections concurrently
   
      private async Task<IEncodedConnection> CreateNatsConnection()
      {
          try
          {
              await _serviceLock.WaitAsync();
              IEncodedConnection connection = new ConnectionFactory().CreateEncodedConnection();
   
              return connection;
          }
          finally
          {
              _serviceLock.Release();
          }
       }
   }
   

Note that both of the solutions can cause latency in your system due the timeout delay/lock.