CODEWITHSUNDEEP
dockergitlab-cigitlab-ci-runner
fliim
fliim

Reputation: 2189

Gitlab runner docker Could not resolve host

Im using 2 containers on my Ubuntu OS: Gitlab-ce and gitlab-runner

Containers names are: gitlab_gitlab_1 and gitlab_gitlab-runner_1

I access to my gitlab app via gitlab.localhost.com:801

I register successfully a runner with this command:

docker exec -it gitlab_gitlab-runner_1 gitlab-runner register --non-interactive --url http://gitlab_gitlab_1 --registration-token _wgMgEx3nBocYQtoi83c --executor docker --docker-image alpine:latest

Then, when I start the job, I got this error message:

Running with gitlab-runner 10.7.1 (b9bba623)
  on 589a617ee407 12ba77f7
Using Docker executor with image alpine:latest ...
Pulling docker image alpine:latest ...
Using docker image sha256:3fd9065eaf02feaf94d68376da52541925650b81698c53c6824d92ff63f98353 for alpine:latest ...
Running on runner-12ba77f7-project-1-concurrent-0 via 01196621a827...
Cloning repository...
Cloning into '/builds/root/test'...
fatal: unable to access 'http://gitlab-ci-token:[email protected]/root/test.git/': Could not resolve host: gitlab.localhost.com
ERROR: Job failed: exit code 1

In both containers, I can access to the hostname gitlab.localhost.com. I think the issue comes from the image alpine which can not resolve the host.

How can I fix that?

Thanks

Edit 1

docker-compose.yml

version: '3'
services:
  gitlab:
    image: 'gitlab/gitlab-ce:latest'
    restart: always
    hostname: 'gitlab.localhost.com'
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://gitlab.localhost.com'
    ports:
      - '801:80'
      - '443:443'
      - '22:22'
    volumes:
      - '/srv/gitlab/config:/etc/gitlab'
      - '/srv/gitlab/logs:/var/log/gitlab'
      - '/srv/gitlab/data:/var/opt/gitlab'
    networks:
      - 'default'

  gitlab-runner:
    image: 'gitlab/gitlab-runner:latest'
    depends_on:
      - 'gitlab'
    restart: always
    volumes:
      - '/srv/gitlab-runner/config:/etc/gitlab-runner'
      - '/var/run/docker.sock:/var/run/docker.sock'
    networks:
      - 'default'
    links:
      - 'gitlab:gitlab.localhost.com'

networks:
  default:
    driver: 'bridge'

Edit 2

docker-compose.yml

version: '3'
services:
  gitlab:
    image: 'gitlab/gitlab-ce:latest'
    restart: always
    hostname: 'gitlab.localhost.com'
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://gitlab.localhost.com'
    ports:
      - '801:80'
      - '443:443'
      - '22:22'
    volumes:
      - '/srv/gitlab/config:/etc/gitlab'
      - '/srv/gitlab/logs:/var/log/gitlab'
      - '/srv/gitlab/data:/var/opt/gitlab'
    networks:
      default:
        aliases:
          - 'gitlab.localhost.com'

  gitlab-runner:
    image: 'gitlab/gitlab-runner:latest'
    depends_on:
      - 'gitlab'
    restart: always
    volumes:
      - '/srv/gitlab-runner/config:/etc/gitlab-runner'
      - '/var/run/docker.sock:/var/run/docker.sock'
    networks:
      - 'default'

networks:
  default:
    driver: 'bridge'

Upvotes: 32

Views: 62936

Answers (12)

Nachiket Upadhye
Nachiket Upadhye

Reputation: 1

I need to do two things.

  1. use dig command to look for the DNS entry and the IP address. and those entry to the /etc/hosts file.
  2. modify the config.toml file. in the following way.
extra_hosts = ["<your link>:<your IP in dig command>"]

These two modification solved my issue.

Upvotes: 0

theruss
theruss

Reputation: 1746

My scenario was very similar: I had a server app whose API needed to be called by another app from within the same gitlab CI job, where both apps ran in separate containers. Curl would result in a "Could not resolve host" error.

Both containers were running in the same network with the --network flag, but the missing part for me was passing the --network-alias flag to docker run on the server's container and then using that name as the host in the api call.

This page provides a useful example of the use of the --network-alias flag.

Upvotes: 0

Raheel Rao
Raheel Rao

Reputation: 31

I solved it by adding network_mode = "host" to config.toml and it picks the network configurations given in /etc/resolve.conf

Upvotes: 2

Taner Mutlu
Taner Mutlu

Reputation: 1

i gat same problem, in my issue server proxy was open and just disable it. #systemctl status firewalld #systemctl stop firewalld

Upvotes: 0

Erokos
Erokos

Reputation: 152

For anyone trying to deploy a runner using a docker executor in a private network with its own dns setup (i.e. having their own dns server), I tried the above solution in specifying the docker network in config.toml and it didn't work. Even though I could resolve gitlab.com from the host and the runner container I kept getting the above mentioned error. So the job container couldn't resolve gitlab.com. In the end what worked for me:

[[runners]]
  ...
  [runners.docker]
    ...
    network_mode = "host"

Upvotes: 4

queeg
queeg

Reputation: 9384

Network-wise a container is like a separate server with it's own ip address, network routes and DNS resolution. The only exception is to run the container on the host network.

That said, you probably need to figure out how DNS resolution works for the host machine (maybe you added something to /etc/hosts?) and then do the same inside the container.

Or use the exception and run on the host network.

Upvotes: 0

Freman Zhang
Freman Zhang

Reputation: 561

This is working for me without define network in docker compose. Just add a extra_hosts record in gitlab-runner config just likes we add A record in other OS.

sudo vim /etc/gitlab-runner/config.toml

  [runners.docker]
tls_verify = false
image = "alpine:latest"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
extra_hosts = ["xxx.xxx.com.cn:192.168.10.110"]

official instruction for this args

https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdocker-section

Upvotes: 12

Riccardo Venturini
Riccardo Venturini

Reputation: 478

I had this problem these days, after trying all the proposed solutions of the various threads, I solved it in the following way:

pkill docker
iptables -t nat -F
ifconfig docker0 down
service docker restart

thanks to My docker container has no internet

Upvotes: 1

fliim
fliim

Reputation: 2189

Thanks to Tarun Lalwan link and according to Joyce Babu post, there are an undocumented option from the gitlab runner repos in the [runners.docker] section

network_mode : Add container to a custom network

So I have to set this option with my network name in the config.toml like

[[runners]]
  ...
  [runners.docker]
    ...
    network_mode = "gitlab_default"

OR when create the runner from command line

docker exec -it gitlab_gitlab-runner_1 gitlab-runner register \
--non-interactive \
--url http://gitlab_gitlab_1 \
--registration-token _wgMgEx3nBocYQtoi83c \
--executor docker \
--docker-image alpine:latest \
--docker-network-mode gitlab_default

Upvotes: 46

Guillaume
Guillaume

Reputation: 406

I know this thread is old, but I saw everywhere threads pointing to this one, but did'nt solve for me.

I got the same error message :

fatal: unable to access 'http://gitlab.maison.fr:82/angular/test1.git/': Could not resolve host: gitlab.maison.fr

Adding network_mode = "host" to config.toml solve the problem.

My config.toml below on my local private network :

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "mars"
  url = "http://gitlab.maison.fr:82/"
  token = "TCfHAheTUdWMU2-fFNxK"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "gitlab/gitlab-runner:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
    network_mode = "host"

Upvotes: 39

Barry Staes
Barry Staes

Reputation: 4115

In case this helps others looking for this..

Same problem but GitLab and GitLab Runner run on different machines in LAN. DNS is working and ping gitlab works, except inside dockers:

Reproduce problem:

$ sudo docker run -it alpine ping gitlab
ping: bad address 'gitlab'
^C

But works with DNS given:

$ sudo docker run -it --dns=172.168.0.1 alpine ping gitlab
PING gitlab (172.168.0.5): 56 data bytes
64 bytes from 172.168.0.5: seq=0 ttl=63 time=0.536 ms
^C

Config actual LAN DNS for docker.

Edit /etc/docker/daemon.json on the GitLab Runner (file did not exist yet) with contents:

{
    "dns": ["172.168.0.1", "1.1.1.1"]
}

Test again, now OK:

$ sudo docker run -it --dns=172.168.0.1 alpine ping gitlab
PING gitlab (172.168.0.5): 56 data bytes
64 bytes from 172.168.0.5: seq=0 ttl=63 time=0.455 ms
64 bytes from 172.168.0.5: seq=1 ttl=63 time=0.905 ms
^C

If this is not how its supposed to be done, i'd be happy to hear.
If this problem shouldnt aught to exist in the first place, i'd be happy to hear as well. I was surprised to not find much references online to this problem for GitLab Runner..

Upvotes: 5

Mostafa Hussein
Mostafa Hussein

Reputation: 11940

As I can see you have defined a network already which means that both gitlab and gitlab-runner are in the same network. you can verify that by using docker inspect. so you need to remove links as you don't need it.

In order to set a network alias you need to change the network part at gitlab service to the following:

gitlab:
  ...
  networks:
    default:
      aliases:
        - gitlab.localhost.com

References:

Upvotes: 0

Related Questions