CODEWITHSUNDEEP
javaasynchronousamazon-web-servicesamazon-s3asyncfileupload
Alexandru Nedelcu
Alexandru Nedelcu

Reputation: 8069

Is the TransferManager in AWS SDK for S3 doing Asynchronous I/O?

I've been reading about TransferManager in the Amazon's AWS SDK for doing S3 uploads, the provided API allows for non-blocking usage, however it's unclear to me if the underlying implementation actually does asynchronous I/O.

I did some reading on the source-code of TransferManager and I cannot understand if the threads in the provided ExecutorService are being blocked or not.

My problem is that if this manager actually does asynchronous I/O without blocking that executor, then I could use the application's global thread-pool that is meant for CPU-bound stuff. So is this actually doing asynchronous I/O or not?

Upvotes: 12

Views: 4739

Answers (1)

Alexandru Nedelcu
Alexandru Nedelcu

Reputation: 8069

After profiling and trying to understand the SDK's source-code I have come to the conclusion that yes, TransferManager does not work asynchronously, because it piggybacks on AmazonS3Client.putObject and such calls, while not blocking the threads per se, go in a loop until the http requests are finished, thus preventing progress in processing the thread-pool's queue.

Upvotes: 14

Related Questions