CODEWITHSUNDEEP
amazon-web-servicesamazon-s3continuous-integrationgitlabdevops
Saif Ali
Saif Ali

Reputation: 427

AWS s3 cp cannot find file from a gitlab build?

I'm working on a Gitlab CI project where i have to push the APK to our aws S3 Bucket for that i have specified the keys in environment variables in the project setting of our repository, now here is my gitlab-ci.yml file:

stages:
  - build
  - deploy

variables:
  AWS_DEFAULT_REGION: us-east-2 # The region of our S3 bucket
  BUCKET_NAME: abc.bycket.info         # bucket name
  FILE_NAME: ConfuRefac.apk

assembleDebug:
  stage: build
  script:
    - export ANDROID_HOME=/home/bitnami/android-sdk-linux
    - export ANDROID_NDK_HOME=/opt/android-ndk
    - export PATH=$PATH:/home/bitnami/android-sdk-linux/platform-tools/
    - export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
    - export PATH=/usr/lib/jvm/java-1.8.0-openjdk-amd64/bin:$PATH
    - chmod +x ./gradlew
    - ./gradlew assembleDebug
    - cd app/build/outputs/apk/debug
    - mv app-debug.apk ${FILE_NAME}
  artifacts:
    paths:
      - app/build/outputs/apk/debug/${FILE_NAME}

deploys3:
  image: "python:latest"  # We use python because there is a well-working AWS Sdk
  stage: deploy
  dependencies:
    - assembleDebug
  script:
    - pip install awscli
    - cd app/build/outputs/apk/debug/
    - ls && pwd
    - aws s3 cp ${FILE_NAME} s3://${BUCKET_NAME}/${FILE_NAME} --recursive

So when the deploy stage starts kicking in it cannot find the file even though in ls you can clearly see that the file is indeed there.

Error

Collecting futures<4.0.0,>=2.2.0; python_version == "2.7" (from s3transfer<0.4.0,>=0.3.0->awscli)
       Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
     Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.16.13->awscli)
       Using cached https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
     Installing collected packages: urllib3, docutils, jmespath, six, python-dateutil, botocore, pyasn1, rsa, futures, s3transfer, PyYAML, colorama, awscli
     Successfully installed PyYAML-5.3.1 awscli-1.18.63 botocore-1.16.13 colorama-0.4.3 docutils-0.15.2 futures-3.3.0 jmespath-0.10.0 pyasn1-0.4.8 python-dateutil-2.8.1 rsa-3.4.2 s3transfer-0.3.3 six-1.14.0 urllib3-1.25.9
     You are using pip version 8.1.1, however version 20.1.1 is available.
     You should consider upgrading via the 'pip install --upgrade pip' command.
     $ cd ${PWD}/app/build/outputs/apk/debug/
     $ ls && pwd
     ConfuRefac.apk
     /home/gitlab-runner/builds/CeGhSYCJ/0/root/confu-android/app/build/outputs/apk/debug
     $ aws s3 cp ${PWD}/${FILE_NAME} s3://${BUCKET_NAME}/${FILE_NAME} --recursive
     warning: Skipping file /home/gitlab-runner/builds/CeGhSYCJ/0/root/confu-android/app/build/outputs/apk/debug/ConfuRefac.apk/. File does not exist.

    Running after_script
    00:00
    Uploading artifacts for failed job
    00:01
     ERROR: Job failed: exit status 1

Upvotes: 0

Views: 675

Answers (1)

Marcin
Marcin

Reputation: 238957

As I indicated in the comments, the issue was caused because --recursive is treating ${FILE_NAME} as a directory, not file.

Which of course would make sense, because one can't recursively copy a single file.

Upvotes: 2

Related Questions