How to run Parallel builds with AWS Codebuild?

Question

I am trying to setup a codebuild where I have one module that would create the dependencies for other builds. And these other builds will use the artifacts generated by the first build and these build should be ran in parallel. I was looking at documentation of CodeBuild it seems there isn't information there. Sample of my buildspec.yml

version: 0.2

#env:
  #variables:
     # key: "value"
     # key: "value"
  #parameter-store:
     # key: "value"
     # key: "value"
  #git-credential-helper: yes

phases:
  #install:
    #If you use the Ubuntu standard image 2.0 or later, you must specify runtime-versions.
    #If you specify runtime-versions and use an image other than Ubuntu standard image 2.0, the build fails.
    #runtime-versions:
      # name: version
      # name: version
    #commands:
      # - command
      # - command
#  pre_build:
#    commands:
#       - mkdir artifacts
#       - pwd
#       - ls
  build:
    commands:
       - cd frontend
       - npm install
       - cd ..
       - cd othermodule
       - npm install
  #post_build:
    #commands:
     #  - cp -a $CODEBUILD_SRC_DIR/frontend/dist. 
      # - command
artifacts:
  files:
     - package-lock.json
     - node_modules/*
  base-directory: frontend
#cache:
  #paths:
    # - paths

Answer 1

From today codebuild support parallel test and splitting of test

https://aws.amazon.com/about-aws/whats-new/2025/01/aws-codebuild-test-splitting-parallelism/

Answer 2

As @MEMark has pointed out, currently AWS CodePipeline service supports Bitbuket.

One suitable approach to reach Bitbucket from AWS is by providing an Star Connection(Developer Tools > Code Build > Settings > Connections).

Once the connection is created the pipeline can be deployed using for instance following code:

AWSTemplateFormatVersion: "2010-09-09"

Description: TBD
Parameters:
  DeployStage:
    Type: String
    Description: name of the stage to deploy(dev, test, prod)
  BranchName:
    Type: String
    Description: TBD 
  CodeStarConnectionARN:
    Type: String
    Description: ARN of the codestar connection to Bitbucket.

Resources:
  
  CodeBuildRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          Effect: Allow
          Principal:
            Service: codebuild.amazonaws.com
          Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AdministratorAccess  

  PipelineRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action: 'sts:AssumeRole'
          Effect: Allow
          Principal:
            Service: codepipeline.amazonaws.com
        Version: '2012-10-17'
      Policies:
        - PolicyName: CodePipelineAccess
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                  - TBD
                  - ...
                Effect: Allow
                Resource: '*'
              -
                Effect: Allow
                Action:
                  - TBD
                  - ...
                Resource: "*"

  BuildUi:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: !Join ['', ['build-ui-', !Ref DeployStage]]
      Description: TBD
      ServiceRole: !Ref CodeBuildRole
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        Type: LINUX_CONTAINER
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/standard:5.0
        EnvironmentVariables:
          - Name: DEPLOY_STAGE
            Type: PLAINTEXT
            Value: !Ref DeployStage
          - Name: DEPLOY_BRANCH_NAME
            Type: PLAINTEXT
            Value: !Ref BranchName    
      Source:
        Type: CODEPIPELINE
        Location: https://user@bitbucket.org/organization/projectName.git 
        BuildSpec: buildspec.ui.yml
      SourceVersion: !Ref BranchName  
      Tags:
        -
          Key: cost
          Value: ui_build
      TimeoutInMinutes: 10

  BuildApi:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: !Join ['', ['build-api-', !Ref DeployStage]]
      Description: TBD
      ServiceRole: !Ref CodeBuildRole
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        Type: LINUX_CONTAINER
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/standard:5.0
        EnvironmentVariables:
          - Name: DEPLOY_STAGE
            Type: PLAINTEXT
            Value: !Ref DeployStage
          - Name: DEPLOY_BRANCH_NAME
            Type: PLAINTEXT
            Value: !Ref BranchName    
      Source:
        Type: CODEPIPELINE
        Location: https://user@bitbucket.org/organization/projectName.git 
        BuildSpec:buildspec.api.yml
      SourceVersion: !Ref BranchName  
      Tags:
        -
          Key: cost
          Value: api_build
      TimeoutInMinutes: 10


  ArtifactStoreBucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: 'BucketOwnerFullControl'
      VersioningConfiguration:
        Status: Enabled    

  NamePipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      ArtifactStore:
        Location: !Ref ArtifactStoreBucket
        Type: S3
      Name: !Join ['', ['pipeline-', !Ref DeployStage]]
      RoleArn: !GetAtt PipelineRole.Arn
      Stages:
        - Name: Source
          Actions:
            - Name: Source
              ActionTypeId:
                Category: Source
                Owner: AWS
                Provider: CodeStarSourceConnection
                Version: '1'
              Configuration:
                ConnectionArn: !Ref CodeStarConnectionARN
                FullRepositoryId: "organization/projectName"
                BranchName: !Ref BranchName                
              OutputArtifacts:
                - Name: project-source
              RunOrder: '1'
        - Name: ParallelBuild
          Actions:
            - Name: BuildUi
              ActionTypeId:
                Category: Build
                Owner: AWS
                Version: 1
                Provider: CodeBuild
              OutputArtifacts:
                - Name: project-build-ui
              InputArtifacts:
                - Name: project-source
              Configuration:
                  ProjectName: !Ref BuildUi
              RunOrder: 1
            - Name: BuildApi
              ActionTypeId:
                Category: Build
                Owner: AWS
                Version: 1
                Provider: CodeBuild
              OutputArtifacts:
                - Name: build-api
              InputArtifacts:
                - Name: project-source
              Configuration:
                  ProjectName: !Ref BuildApi
              RunOrder: 1                   

Notes:

• it assumes the repository has this git url: https://user@bitbucket.org/organization/projectName.git
• CodeBuildRole and PipelineRole should be defined properly for the use case

Answer 3

CodeBuild is used to automate build step, not to automate the whole CICD process. In CodeBuild, you specify buildspec.yml to automate a sequence of steps that need to be performed in that particular build.

If you need to automate sequence of builds then the easiest option that you have is to use CodePipeline where you can create stage for each step in your CICD process. In your case, this would mean that one step (stage) would be CodeBuild action that you have described in your post which would transition into another stage where you can specify another CodeBuild actions and these actions can be specified to take artifacts from the previous step as an input and you can run them in parallel.

So it would look like this

INPUT -> STAGE (perform initial build) -> STAGE (specify multiple build actions in parallel - in console, this can be done by placing them next to each other horizontally instead of vertically)

The other option, without using CodePipeline, would be to use Lambda function with CloudWatch events. CodeBuild publishes event once the build is done. You can subscribe Lambda function to that event and write a code that would execute following builds as needed.