CODEWITHSUNDEEP
javamultithreadingamazon-kinesisaws-lambda
AKSHAY SHINGOTE
AKSHAY SHINGOTE

Reputation: 407

AWS Lambda Java Multithreading

Hello I have made an AWS Lambda function for Kinesis stream for batch size of 100 & I am trying to execute it in multi-threaded environment but the problem is that in multi-threaded environment,it works very slow compare to single-threaded .. Like I can share you the numbers : For 512 MB with 60 seconds timeout,Lambda function executes 955 records in 684 ms & 1031 records in 435 ms & For multithreaded it executes 430 records in 878808 ms & 433 records in 893862 ms for same memory (i.e. 512 MB & 60 second timeout)

Following is my Lambda function code which is executing in multi-threaded behaviour:

public String myHandler(final KinesisEvent kinesisEvent, final Context context) {
      Thread thread = new Thread(new Runnable(){

            //@Override
            public void run() {
                int singleRecord=0;
                long starttime=System.currentTimeMillis();
                //LambdaLogger lambdaLogger=context.getLogger();
                for(KinesisEventRecord rec : kinesisEvent.getRecords())
                {
                    singleRecord=0;
                    System.out.println("Kinesis Record inside is:"+new String(rec.getKinesis().getData().array()));
                    //count++;
                    singleRecord++;
                    //  System.out.println(new String(rec.getKinesis().getData().array()));
                }
                count=count+singleRecord;
                long endtime=System.currentTimeMillis();
                long totaltime = endtime-starttime;
                time=time+totaltime;
                System.out.println("Time required to execute single Lambda function for "+singleRecord+" records is"+" :: "+totaltime+" milliseconds");
                System.out.println("Total time required to execute Lambda function for "+count+" records is"+" :: "+time+" milliseconds");
            }
        });
        thread.start();
        return null;
    } //end of handler method

Does Lambda function executes slow in multithreaded environment? I want to know what might be the reason behind the slow processing of this multithreaded Lambda function? If I want this function to work faster than single-threaded function then what changes should I do in this code?

Upvotes: 4

Views: 6841

Answers (1)

Geoff
Geoff

Reputation: 1287

Two further options:

  1. Have your initial Lambda use the SDK to asynchronously launch further lambdas that actually do work (fan out)
  2. Use Lambda scheduling with scheduling offsets and long timeouts, so that you have multiple lambdas running concurrently.

Upvotes: 2

Related Questions