CODEWITHSUNDEEP
dockerdocker-swarmdocker-network
borchero
borchero

Reputation: 6002

Docker Network not Found

In our team, we are currently transitioning to Docker to deploy everything on our server.

We are using Docker Swarm and multiple (10+) compose files defining plenty (20+) of services. Everything works beautifully so far, except when we take down our stack using docker stack rm <name> (and redeploy using docker stack deploy <options> <name>): about every second time, we get the following error:

Failed to remove network <id>: Error response from daemon: network <id> not foundFailed to remove some resources from stack: <name>

When using docker network ls, the network is indeed not removed, however, docker network rm <id> always results in the following:

Error response from daemon: network <id> not found

What makes that even more strange is the fact that docker network inspect <id> returns a normal output. The networks are always overlay networks that are created with the compose files used to deploy our stack. Currently, we only have a single node in our Swarm.

Our current "workaround" is to restart Docker (which resolves the issue), but that is not a viable solution in a production environment. Leaving the swarm and joining it again does not resolve the issue either.

At first, we thought that this issue is related to Docker for Mac only (as we first encountered the issue on local machines), however, the same issue arises on Debian Stretch. In both cases, we use the latest Docker distribution available.

I would really appreciate any help!

Upvotes: 141

Views: 163180

Answers (12)

DevMinds
DevMinds

Reputation: 285

At some point, none of the proposed solutions worked for me. Of course after:

  • docker compose -f dev-compose.yml down --volumes --remove-orphans echo "y"|docker network prune and docker compose -f dev-compose.yml up -d --force-recreate --build

Then I realized, when we create project, usually, we don't specify name of the project or in this case, the network. So the network name we give to services is prepended by the docker-compose root folder name (ex amazing_app) and the network name (ex: app_network) which will give: amazing_app_app_network as the network name.

So, withouth specifying the network name in the network section of the docker compose file:

  • This will work: docker network inspect amazing_app_app_network
  • This won't: docker network inspect app-network unless you give a name to the network.

So, when we run a command like: docker network inspect app-network we won't find it and have an answer like: [] Error response from daemon: network app-network not found

Let say I have a project with a docker-compose file:

services:
  mongo:
    image: mongo:7.0
    container_name: mongo
    restart: always
    volumes:
      - "mongodb_data:/data/db"
    ports:
      - 27017:27017
    environment:
      - MONGO_INITDB_ROOT_USERNAME=root
      - MONGO_INITDB_ROOT_PASSWORD=password
    networks:
      - app-network

  nginx:
    image: nginx:latest
    container_name: nginx
    restart: always
    ports:
      - "8081:80"
    volumes:
      - ./nginx/nginx-dev.conf:/etc/nginx/conf.d/default.conf
    depends_on:
      - app
    networks:
      - app-network
    healthcheck:
      test: [ "CMD", "curl", "-f", "http://app:3000" ]
      interval: 10s
      timeout: 5s
      retries: 3

  app:
    image: node:20.17.0
    container_name: app
    restart: always
    volumes:
      - "./:/opt/app"
    working_dir: '/opt/app'
    env_file:
      - .env
    depends_on:
      - mongo
    command: /bin/sh -c "npm run dev"
    networks:
      - app-network

networks:
  app-network:
    name: app-network # I specified this line to be able to find my network by its name
    driver: bridge

volumes:
  mongodb_data:

Upvotes: 1

Mukesh Chapagain
Mukesh Chapagain

Reputation: 25948

In my case, my container was linked to two different networks.

Found it by inspecting the container:

docker inspect <my-container-name>

Two networks were showing under NetworkSettings > Networks in the output JSON.

I disconnected the wrong network from the container:

docker network disconnect <wrong-network-name> <my-container-name>

Then, the issue got resolved and I was able to run my container.

Upvotes: 0

Phillip Dawson
Phillip Dawson

Reputation: 111

None of the above worked for me. I've used the following docker-compose command to fix.

docker-compose up --force-recreate --remove-orphans

Upvotes: 11

FantomX1
FantomX1

Reputation: 1711

To me it helped to use

docker network connect existingNetwork container

for all network failing containers, nothing else seemed to help to me, since they seemed to be connected as implied to a non existing network, here mentioned disconnecting them dindt help only re/connecting to existing, and removing network that non-existing ended up in an error "could not remove a non existing network" ofc

Upvotes: 5

Tim
Tim

Reputation: 90

This is the experience I got and I think it might help. Docker network is capable of doing bridging. In course told, a container can disconnect and connect from one to the other. If one disconnects from current and connect to the other, and the current disappear due to shutdown/network prune, the independent container will lose the connection. Later, when you try to start, only found "network not found" error.

The solution to this is start swarm/cluster (in my case I start with docker-compose up), disconnect the container (even it's yet up) from that network using force (-f). Connect back to that (different ID, but same name) network. Now you can successfully start it without "network not found" error. So, the point is it maybe happens to see same name and different ID network.

Upvotes: 0

Ashim
Ashim

Reputation: 473

I could not get rid of the networks by any of the methods in previous answers.

This is what worked for me.

systemctl restart docker

Upvotes: 10

Jinna Baalu
Jinna Baalu

Reputation: 7809

Deleting the "network not found" in docker

Inspect the network which we are unable to delete

docker network inspect <id> or <name>

Disconnect the network

docker network disconnect -f <networkID> <endpointName> or <endpointId>

Delete unused networks

docker network prune

Upvotes: 20

oDinZu
oDinZu

Reputation: 328

After using the docker prune command, I was unable to launch the docker container on a network.

The following errors stated:

ERROR: for jekyll-serve Cannot start service jekyll-serve: network b52287167caf352c7a03c4e924aaf7d78e2bc372c703560c003acc758c013432 not found ERROR: Encountered errors while bring up the project.

docker system prune enabled me to begin using docker-compose up again.

More info here: https://docs.docker.com/config/pruning/

Upvotes: 22

joaoluiznaufel
joaoluiznaufel

Reputation: 37

old containers are still using old network. Probably you removed networks but forgot to rm old containers. Just remove old containers, create your network and build again.

Upvotes: 0

Nick Beaird
Nick Beaird

Reputation: 3339

If you are attempting to add a container to an existing network that no longer exists, then you can use docker-compose up --force-recreate. I found this GitHub issues comment to be a helpful overview.

Upvotes: 324

fcnorman
fcnorman

Reputation: 1186

You can always use docker system prune -a to get rid of the old network. This will not delete your volumes.
It will take longer to docker-compose up --build -d the next time, but it will get you past your current problem.

Upvotes: 30

Julio Daniel Reyes
Julio Daniel Reyes

Reputation: 6365

That sounds exactly like this issue.

Stack rm followed "too fast" by stack deploy would race for the creation/removal of networks, possibly other stack resources.

The issue is still open as of today (docker/cli), but you could try the workaround suggested:

until [ -z "$(docker service ls --filter label=com.docker.stack.namespace=$COMPOSE_PROJECT_NAME -q)" ] || [ "$limit" -lt 0 ]; do
  sleep 1;
done

until [ -z "$(docker network ls --filter label=com.docker.stack.namespace=$COMPOSE_PROJECT_NAME -q)" ] || [ "$limit" -lt 0 ]; do
  sleep 1;
done

Upvotes: 6

Related Questions