CODEWITHSUNDEEP
javamultithreadingamazon-s3aws-sdk
Dimon
Dimon

Reputation: 803

Asynchronous multiple files upload to AWS S3 bucket

I want to upload multiple files to AWS S3 bucket with transfer manager from AWS Java SDK (aws-java-sdk-s3-1.11.66.jar), so my function over here:

public static void uploadDirectory(String virtualDirectoryKeyPrefix, File directory) {
    try {
        MultipleFileUpload multipleFileUpload = instance.transferManager.uploadFileList(
                instance.bucketName,
                virtualDirectoryKeyPrefix,
                directory,
                Arrays.asList(Objects.requireNonNull(directory.listFiles())));

        multipleFileUpload.waitForCompletion();
    } catch (InterruptedException e) {
        Logger.error("Not able to write to s3 bucket : " + virtualDirectoryKeyPrefix, e);
    }
}

The problem that it works only when I call the 'waitForCompletion' method and it makes my function synchronous because in this case, I wait until the operation finish. I expected that it must work without 'waitForCompletion' call, but it not, could U please help me? Thanks.

Upvotes: 0

Views: 3051

Answers (2)

Sathya Narayan C
Sathya Narayan C

Reputation: 154

Why can't you add a listener to the transferManager instance 

    transferMangerInstance.upload(putObjectRequestInstance);
    transferMangerInstance.addProgressListener(ProgressListener progressListener);

    transferMangerInstance.addProgressListener(new ProgressListener() {
                @Override
                public void progressChanged(ProgressEvent progressEvent) {
                    if(progressEvent.getEventType() == ProgressEventType.TRANSFER_COMPLETED_EVENT) {
                  //Do your completion logic here
                    }
                }
            }

Upvotes: 1

Andrei-Nicolae Grosu
Andrei-Nicolae Grosu

Reputation: 61

waitForCompletion only blocks until the transfer is complete(https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/Transfer.html#waitForCompletion--).

I suspect your transfer was working also before but because it is asynchronous you were not checking when it was done. In order to receive asynchronous updates add a ProgressListener as described here: https://docs.aws.amazon.com/AmazonS3/latest/dev/HLTrackProgressMPUJava.html

Upvotes: 0

Related Questions