CODEWITHSUNDEEP
gitlabgitlab-cigitlab-ci-runner
Majid Parvin
Majid Parvin

Reputation: 5012

gitlab: Runner is offline, last contact was about some hours ago

recently my runners have been stopped and I don't know why?

I've just upgraded nodejs on the server and it did happen.

after this problem, I've tried to update gitlab to the latest version and check the runner status but the problem still persists and in the title of grey icon shows:

Runner is offline, the last contact was about 22 hours ago.

Runner is offline, the last contact was about 22 hours ago

What should I do?

and when I try to Retry stuck jobs, see this error:

This job is stuck, because you don't have any active runners online with any of these tags assigned to them: 'my label'.

Any Help is appreciated!

Upvotes: 41

Views: 57246

Answers (13)

Venkat Bhupathi Konathala
Venkat Bhupathi Konathala

Reputation: 11

for me its worked just run below command:

sudo gitlab-runner run

Upvotes: 1

kenzie
kenzie

Reputation: 2701

For gitlab 15.7.1 and 15.8.0, this is what I did to resolve disconnecting runners symptoms:

  • make sure home directory exists, create if it doesn't:

    sudo mkhomedir_helper gitlab-runner

  • make sure --working-directory is set to gitlab-runner home in /etc/systemd/system/gitlab-runner.service

  • running stop told me to run daemon-reload and then restarted the service:

    sudo service gitlab-runner stop
sudo systemctl daemon-reload
sudo service gitlab-runner stop
sudo service gitlab-runner start
sudo service gitlab-runner status

Upvotes: 0

Frak
Frak

Reputation: 865

In my case, after upgrading from an old gitlab-runner (9.5.1 -> 11.11.4) I couldn't get my runner to work. Even after removing it and registering a new one. I noticed that my config.toml file didn't have the token that matches the one in the Runner CI/CD Settings page for my new runner.

I did the following to get it to work:

  1. Remove the /etc/gitlab-runner/config.toml file (perhaps unnecessary)

  2. Click on Reset runners registration token in Gitlab under project's CI/CD Settings -> Runner page while registering a new runner (gitlab-runner register)

  3. Continue with runner registration as usual.

After that, my runner was picked up and worked as usual.

Upvotes: 1

Wipster
Wipster

Reputation: 1570

For me the problem was, that the URL of our self-hosted GitLab changed. I had to change the URL of the runners manually:

sudo vi /etc/gitlab-runner/config.toml

After restarting, I could verify that all runners were back online:

sudo gitlab-runner restart
sudo gitlab-runner verify

Upvotes: 3

Peter F
Peter F

Reputation: 570

Check gitlab-runner logs with journalctl -u gitlab-runner -f

For me the problem was:

gitlab-runner[2023]: FATAL: Service run failed                           error=chdir /var/lib/gitlab-runner: no such file or directory

so created the folder and restarted gitlab-runner 

sudo mkdir /var/lib/gitlab-runner
sudo gitlab-runner restart

probably something to do with installation not run by root user

Upvotes: 3

Riccardo
Riccardo

Reputation: 1173

This worked for me:

sudo gitlab-runner verify

since gitlab-runner restart didn't actually restarted runners

Upvotes: 27

Firecow
Firecow

Reputation: 591

Sorry for resurrecting this old thread.

I'm currently working on making a gitlab runner that works locally. Still in the early phases, but eventually it will become very relevant. It doesn't seem like gitlab want/have time to make this, so here you go. https://github.com/firecow/gitlab-runner-local

Upvotes: 0

Amir Kaftari
Amir Kaftari

Reputation: 1514

after restart gitlab-runner , you can use this for new register runner : 

gitlab-runner restart

gitlab-runner register --name my-runner --url http://gitlab.example.com --registration-token my-registration-token

Upvotes: 1

f3xy
f3xy

Reputation: 775

If you runner is run under systemd

journalctl -u gitlab-runner.service

may prove enlightening. It did in my case.

gitlab-runner[18449]: FATAL: chdir /home/gitlab-runner: no such file or directory

Upvotes: 7

Swapnil M Mane
Swapnil M Mane

Reputation: 336

Sharing my experience for future readers.

In my case, the internet was not working on the machine in which gitlab-runner was installed.

I have fixed the internet issue on my gitlab-runner's machine and everything works as expected.

Thanks!

Upvotes: 3

JSeven
JSeven

Reputation: 655

I had the same error but using gitlab runner on GKE. To fix it I just deleted the gitlab-runner pod. A new pod was initialized and the connection was up again.

Upvotes: 0

fedorqui
fedorqui

Reputation: 289725

To me, the following solved the problem:

gitlab-runner restart

Where gitlab-runner is a symlink to gitlab-ci-multi-runner:

GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with GitLab CI, the open-source continuous integration service included with GitLab that coordinates the jobs.

Upvotes: 34

Majid Parvin
Majid Parvin

Reputation: 5012

I don't why but I found the gitlab-runner was completely out of service. I've tried to fix it. at the first step, I've updated the gitlab-runner bit it didn't work. after that I decided to reinstall the gitlab-runner so I've met this steps:

  1. removing the gitlab-runner user with this command line: sudo deluser --remove-home gitlab-runner
  2. following the manual installation and configuration on here.

and it worked.

Upvotes: 4

Related Questions