Scalable video encoding?

Question

I have around 10000-20000 PNG images that I need to combine to form a video. Using ffmpeg works well, but doesn't scale at the current stage. Each encoding process takes around five minutes and consumes 800-900 megabytes of RAM on a machine made to process web requests and everything slows down. I am looking for a cheap alternative to process these videos in bulk. Things I've considered:

• AWS lambda, but that ran out of disk space and I ran over the five minute limit on longer videos
• AWS EC2, but how would I make that cost-effective? Spin up a new instance while I need to encode, then shut it down?
• AWS elastic transcoder, but that doesn't deal with images
• buy a dedicated server, but that's too expensive

Are there any other services or possibilities to generate the videos cost-effectively? Perhaps an AWD service I've overlooked?

Answer 1

Can't you split your jobs in multiple Lambda functions?

If not, I think you can achieve this by mixing some of the services you mention.

S3 input buquet (PNG images zipped) --> triggers lambda functions --> puts SQS message/task --> consumed by EC2 spot intances.

Important point:

• Make the EC2 jobs to be gracefully stopped if your bid price is lower than current and AWS turn off your instance.
• Choose the appropiate EC2 instance family and the number of them you need