CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-codebuild
Viet
Viet

Reputation: 6953

AWS CodeBuild buildspec.yml get all files and subfolders recursively

I'm trying to use AWS CodeBuild to get all files and subfolders inside a nested public folder and deploy to an S3 bucket using CodePipeline. I was able to hook them all together but struggling to configure the buildspec.yml file to get the output I want.

My folder structure:

<path>/public/

├── 404.html
├── css
│   ├── ...
├── fonts
│   ├── bootstrap
│   │   ├── ...
│   ├── icomoon
│   │   ├── icomoon
│   │   │   ├── ...
│   └── simple-line-icons
│       ├── ...
├── images
│   ├── ...
├── index.html
├── index.xml
├── js
│   ├── ...
└── tags
    └── index.xml

I need to put everything (including the subfolders) inside the public folder into the root of an S3 bucket.

So far I've tried following the docs here, here and here. I've tried using:

Upvotes: 30

Views: 49376

Answers (6)

Arkady
Arkady

Reputation: 11

For me a very simple construct like this was enough:

artifacts:
  files:
    - 'dist/**/*'

Upvotes: 1

Alex Christodoulou
Alex Christodoulou

Reputation: 2963

For those who are facing this issue while using CodeBuid in CodePipline and have tried the correct configuration for buildspec.yml file, the culprit for me was on the deploy stage, where I had set as Input Artifact the Source artifact instead of the Build artifact.

Upvotes: 1

Sergei Lissovski
Sergei Lissovski

Reputation: 551

TL;DR You probably don't need to use "discard-paths: yes". So, drop "discard-paths: yes" and instead use "base-directory" with "**/*" for glob pattern. Read on for more reasoning why.

So yeah, as it usually happens, the culprit was pretty trivial. In my case the directory structure looked like this:

dist/
|-- favicon.ico
|-- index.html
|-- js
|   `-- app.js
`-- revision.txt

So in order for the artifact to contain everything from the "dist" directory whilst preserving the nested directories structure, the buildspec had to look like this:

artifacts:
  files:
    - '**/*'
  base-directory: 'dist'

Essentially "base-directory" parameter behaves as unix's "cd" command, so at first CodeBuild jumps to that directory and only after that resolves the glob pattern, in this case - **/*. If you also happen to use "discard-paths" then what happens is that, presumably, CodeBuild will change current directory to "dist", resolve the files structure correctly but then remove all path prefixes from the resolved paths, thus the result will be a plain directory structure with no nesting.

Hope that helps!

Upvotes: 55

ReedAccess
ReedAccess

Reputation: 59

This worked for me.

    /
    buildspec.yml
    /public
        index.html
        /js
        /img
        appspec.yml
    /src

in buildspec.yml

    artifacts:
    files:
      - 'public/*'
    discard-paths: yes

Upvotes: 1

Harmanpreet Singh Jassal
Harmanpreet Singh Jassal

Reputation: 41

I was running into a similar issue and after many permutations below syntax worked for me.

artifacts:
  files:
    - './**/*'
  base-directory: public
  name: websitename

Upvotes: 4

Viet
Viet

Reputation: 6953

I've figured out a way to work around it. I don't think it's the best way but it works. I hope you can find this solution useful buildspec.yml:

version: 0.2

phases:
  build:
    commands:
      - mkdir build-output
      - cp -R <path>/public/ build-output
  post_build:
    commands:
      - mv build-output/**/* ./
      - mv build-output/* ./
      - rm -R build-output *.yml LICENSE README* .git*
artifacts:
  files:
    - '**/*'

In words:

  • I copy everything inside the nested <path>/public folder out to a folder called ./build-output.

  • Then in post_build I move everything out of folder build-output.

  • Delete files pulled from my GitHub's repo which aren't needed to host the static website in the S3 bucket

  • Then I get the result I want: all files inside public in the root of the S3 bucket with the right folder tree.

Update:

  • You can also use my buildspec.yml example file here. Since the example is out of the scope of this question, I don't paste the code here.
  • I explain in details here.

Upvotes: 22

Related Questions