CODEWITHSUNDEEP
androidcontinuous-integrationgitlab-ci
inlacou
inlacou

Reputation: 166

Gitlab-ci stage not finishing on build failed

I have gitlab ci working with unit tests (whether they fail or not) and instrumentation tests (when they do not fail), but when ./gradlew connectedAndroidTest fails because some test failed, the command seems not to return anything to gitlab ci, and so gitlab ci waits until timeout.

I have tried parameters on ./gradlew connectedAndroidTest command and none worked.

stages:
  - tests
  - cleanup
  - quality #not used here
  - distribute #not used here

unit:
  stage: tests
  before_script:
    - bundle install
  script:
    - bundle exec fastlane unit_tests
  only:
    - master
    - tags
  tags:
    - cimacmini
  artifacts:
    name: "reports_${CI_PROJECT_NAME}_${CI_COMMIT_REF_NAME}"
    expire_in: 60 days
    paths:
      - app/build/reports/tests/

instrumentation:
  stage: tests
  only:
    - master
    - tags
  tags:
    - cimacmini
  script:
    - emulator @instrumentation &
    - android-wait-for-emulator.sh
    - adb devices
    - adb shell settings put global window_animation_scale 0 &
    - adb shell settings put global transition_animation_scale 0 &
    - adb shell settings put global animator_duration_scale 0 &
    - adb shell input keyevent 82 &
    - ./gradlew --status
    - ./gradlew connectedAndroidTest --continue
    - stop-emulators.sh
  artifacts:
    name: "reports_${CI_PROJECT_NAME}_${CI_COMMIT_REF_NAME}"
    expire_in: 60 days
    paths:
      - app/build/reports/androidTests/connected/

closeemulators:
  stage: cleanup
  only:
    - master
    - tags
  tags:
    - cimacmini
  script:
    - stop-emulators.sh
  when: always

So stop-emulators.sh is never called if some instrumental test fails. Not from the instrumentation step, nor from the closeemulators step. The gitlab console displays this:

BUILD FAILED in 6m 17s

58 actionable tasks: 58 executed

And waits forever (until forced timeout). Has someone an idea of what to do?

Upvotes: 2

Views: 1828

Answers (2)

Artem Chernousov
Artem Chernousov

Reputation: 509

I've also faced this problem and here is my solution: According to the manual https://docs.gitlab.com/ee/ci/yaml/README.html#script we can use this syntax:

job:
  script:
    - false || exit_code=$?
    - if [ $exit_code -ne 0 ]; then echo "Previous command failed"; fi;

and save last command exit code to the "exit_code" variable and use it further. Here is my test stage from gitlab-ci.yml:

test:
  stage: test
  tags:
    - android
    - osx
  script:
    - $ANDROID_HOME/emulator/emulator -avd Pixel_XL_API_29 -wipe-data -no-window -no-audio & EMULATOR_PID=$!
    - chmod +x ./ci/android-wait-for-emulator.sh
    - ./ci/android-wait-for-emulator.sh $ANDROID_HOME/platform-tools
    - ./gradlew cAT || exit_code=$?
    - if [ $exit_code -ne 0 ]; then echo "Previous command failed"; fi;
    - chmod +x ./ci/stop-emulators.sh
    - ./ci/stop-emulators.sh $ANDROID_HOME/platform-tools
    - if [ $exit_code -ne 0 ]; then exit 1; fi;

So, if the command "./gradlew cAT" fails, job continues and stop-emulators.sh will be executed, BUT job will be succeeded in the end, which is not correct, thats why we need the last row to exit job with failure and mark job as "failed" in the pipeline.

- if [ $exit_code -ne 0 ]; then exit 1; fi;

So, what I have in the end https://i.sstatic.net/fEU9Z.png

Hope, this helps.

Upvotes: 3

Maciej Pigulski
Maciej Pigulski

Reputation: 1562

During our development of UI integration tests we have stumbled upon same mind-boggling problem. Reason for this behaviour was that when you ask gitlab to execute a script it will monitor all the process it has launched during that script and it will wait for them to finish in order to finish this step. One of that steps is starting an emulator.

In case of tests failure stop-emulators.sh won't execute because for gitlab ./gradlew connectedAndroidTest --continue is the indication to fail the whole pipeline and it is waiting for emulator to stop. But emulator won't stop as next step that stops the emulator won't be executed at all. This is a kind of a deadlock that resolves into a timeout.

If you log in to the machine that is running the test and kill the emulator manually you will see that pipeline will get unblocked and gitlab will finish the execution reporting a test error.

Solution was to move ./gradlew connectedAndroidTest --continue to after_script section of .gitlab-ci.yml that is executed also after a failed build.

Documentation: https://docs.gitlab.com/ee/ci/yaml/#before_script-and-after_script

Upvotes: 0

Related Questions