CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-ec2aws-lambda
connor449
connor449

Reputation: 1679

AWS service for video post processing

I have a local program that inputs a video, uses a tensorflow model to do object classification, and then does a bunch of processing on the objects. I want to get this running in AWS, but there is a dizzying array of AWS services. My desired flow is:

video gets uploaded to s3 --> do classification and processing on each frame of said video --> store results in s3.

I've used Lambda for similar work, but this program relies on 2 different models and its overall size is ~800 MB.

My original thought is to run an ec2 instance that can be triggered when 3 receives a video. Is this the right approach?

Upvotes: 0

Views: 249

Answers (3)

Shelly Dar Rapaport
Shelly Dar Rapaport

Reputation: 96

You can use Amazon Elemental to split the video file, and distribute the parts to different lambdas, so you can scale it, and process it in parallel.

Upvotes: 1

Bartłomiej Gładys
Bartłomiej Gładys

Reputation: 4615

You can consider creating a docker image containing your code, dependencies, and the model. Then, you can push it to ECR and create a task definition and fargate cluster. When the task definition is ready, you can set up a cloudwatch event, which will be triggered upon s3 upload, and as a target, you can select fargate resources that were created at the beginning.

There's a tutorial with a similar case available here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatch-Events-tutorial-ECS.html

Upvotes: 3

Mark B
Mark B

Reputation: 201088

I think you're on the right track. I would configure S3 to send new object notifications to an SQS queue. Then you can have your EC2 instance poll the queue for pending tasks. I would probably go with ECS + Fargate for this, but EC2 also works.

Upvotes: 2

Related Questions