CODEWITHSUNDEEP
amazon-web-servicesaws-codebuildaws-batch
santos82h
santos82h

Reputation: 470

AWS Batch vs AWS CodeBuild

I am very new in AWS and when I was searching something to download a code from GitHub (a python project), run it, and save the output in s3 the first service that I found was CodeBuild.

So I implement this kind of workflow using CodeBuild.

But now I have seen that AWS have a service called AWS Batch and I am wondering if I should migrate my arquitecture to AWS Batch.

Can you explain which one - AWS CodeBuild or AWS Batch - is more suitable with my case? When use AWS Batch instead of AWS CodeBuild?

Thank very much.

Upvotes: 4

Views: 1704

Answers (2)

TrieuNomad
TrieuNomad

Reputation: 1820

TLDR Summary: AWS Codebuild is the nicer choice for simple jobs.

My (reverse) experience...

I needed to run a simple job that pulls data from external api, read/write to external database, and generate a CSV report.

The job takes ~1 hour to run, so AWS Lambda is out of the picture.

After some googling, I found AWS Batch and decided to give Creating a Simple “Fetch & Run” AWS Batch Job a try.

The required steps to this "simple" job working:

  • Build a Docker image with the fetch & run script
  • Create an Amazon ECR repository for the image
  • Push the built image to ECR
  • Create a simple job script and upload it to S3
  • Create an IAM role to be used by jobs to access S3
  • Configure a compute environment
  • Create a job queue
  • Create a job definition that uses the built image
  • Submit and run a job that execute the job script from S3

After spending the time to create all these resources, it did not work out of the box. I found myself debugging random things I shouldn't have to debug such as:

  • Dockerfile
  • entrypoint script
  • ECS cluster
  • EC2 instance and autoscaling group

After failing to find simple practical examples, and realizing the amount of effort required, I decided to explore other solutions.

I stumbled onto Using AWS CodeBuild to execute administrative tasks and this post.

I've used AWS Codebuild in the past for CI/CD pipeline, and thought "what the hell, lets give it a try". In a shorter amount of time, I was able to get a "codebuild job" running on a cloudwatch scheduler and codebuild slack notifications added, with less effort:

  • Connect build project to your source code
  • Select a runtime environment
  • Create IAM role
  • Create a buildspec.yml and add runtime commands

One major advantage is that CodeBuild runs tasks in a full-blown Linux environment.

Drawbacks:

  • Max execution time of 8 hours

AWS Codebuild was much easier to get working for my simple job.

Sorry for the long post, just wanted to share my experience with these 2 services.

Upvotes: 6

Marcin
Marcin

Reputation: 238249

AWS Batch is used highly parallel computations, e.g., processing large number of images at the same time:

AWS Batch enables you to run batch computing workloads on the AWS Cloud. Batch computing is a common way for developers, scientists, and engineers to access large amounts of compute resources, and AWS Batch removes the undifferentiated heavy lifting of configuring and managing the required infrastructure, similar to traditional batch computing software.

Thus its not suited for what you are trying to use it. CodeBuild is better choice, based on your description.

Upvotes: 4

Related Questions