CODEWITHSUNDEEP
gitlabgitlab-ci-runner
Alex
Alex

Reputation: 44275

How to get immediate output from a job run within gitlab-runner?

The command gitlab-runner lets you "test" a gitlab job locally. However, the local run of a job seems to have the same problem as a gitlab job run in gitlab CI: The output is not immediate!

What I mean: Even if your code/test/whatever produces printed output, it is not shown immediately in your log or console.

Here is how you can reproduce this behavior (on Linux):

  1. Create a new git repository

     mkdir testrepo
 cd testrepo
 git init

  2. Create file .gitlab-ci.yml with the following content

      job_test:
   image: python:3.8-buster
   script:
     - python tester.py

  3. Create a file tester.py with the following content:

     import time

 for index in range(10):
     print(f"{time.time()}  test output")
     time.sleep(1)

  4. Run this code locally

     python tester.py

which produces the output

1648130393.143866  test output
1648130394.1441162  test output
1648130395.14529  test output
1648130396.1466148  test output
1648130397.147796  test output
1648130398.148115  test output
1648130399.148294  test output
1648130400.1494567  test output
1648130401.1506176  test output
1648130402.1508648  test output

with each line appearing on the console every second.

  1. You commit the changes

     git add tester.py
 git add .gitlab-ci.yml
 git commit -m "just a test"

  2. You start the job within a gitlab runner

     gitlab-runner exec docker  job_test

 ....
 1648130501.9057398  test output
 1648130502.9068272  test output
 1648130503.9079702  test output
 1648130504.9090931  test output
 1648130505.910158  test output
 1648130506.9112566  test output
 1648130507.9120533  test output
 1648130508.9131665  test output
 1648130509.9142723  test output
 1648130510.9154003  test output
 Job succeeded

Here you get essentially the same output, but you have to wait for 10 seconds and then you get the complete output at once!

What I want is to see the output as it happens. So like one line every second.

How can I do that for both, the local gitlab-runner and the gitlab CI?

Upvotes: 2

Views: 6059

Answers (3)

Said Alizadeh
Said Alizadeh

Reputation: 11

The -u option for python command does the job.

Add the option -u (unbuffered) to your python command in your yaml file.

script: - python -u tester.py

Upvotes: 1

Lubo
Lubo

Reputation: 1827

You can poll gitlab web for new log lines as fast as you can:

For running job, use url like: https://gitlab.example.sk/grpup/project/-/jobs/42006/trace It will send you a json structure with lines of log file, offset, size and so on. You can have a look at documentation here: https://docs.gitlab.com/ee/api/jobs.html#get-a-log-file

Sidenote: you can use undocumented “state” parameter from response in subsequent request to get only new lines (if any). This is handy.

Through, this does not affect latency of arrival newlines from actual job from runner to gitlab web/backend. See sytech answer for this question.

This answer should help, when there is configured redis cache, incremental logging architecture, and someone wants to get logs from currently running job in "realtime". Polling is still needed through.

Some notes can be found also on forum: https://forum.gitlab.com/t/is-there-an-api-for-getting-live-log-from-running-job/73072

Upvotes: 0

sytech
sytech

Reputation: 40861

In the source code, this is controlled mostly by the clientJobTrace's updateInterval and forceSendInterval properties.

These properties are not user-configurable. In order to change this functionality, you would have to patch the source code for the GitLab Runner and compile it yourself.

The parameters for the job trace are passed from the newJobTrace function and their defaults (where you would need to alter the source) are defined here.

Also note that the UI for GitLab may not necessarily get the trace in realtime, either. So, even if the runner has sent the trace to GitLab, the javascript responsible for updating the UI only polls for trace data every ~4 or 5 seconds.

Upvotes: 3

Related Questions