CODEWITHSUNDEEP
javaamazon-web-servicesamazon-s3
Abhishek
Abhishek

Reputation: 107

How to Download entire Folder located on S3 Bucket?

I have used Java SDK and try to download Folder using GetObjectRequest class, but it is possible to download my folder incuding its subFolder and all files to download ?

Upvotes: 7

Views: 25761

Answers (4)

Reece
Reece

Reputation: 691

Here's the code to download a whole bucket (somewhat tested):

import com.amazonaws.AmazonServiceException;
import aws.example.s3.XferMgrProgress;
import com.amazonaws.services.s3.transfer.TransferManager;
import com.amazonaws.services.s3.transfer.TransferManagerBuilder;
import com.amazonaws.services.s3.transfer.MultipleFileDownload;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import java.io.*;
import com.amazonaws.auth.PropertiesFileCredentialsProvider;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.AmazonClientException;
public class S3DownloadApp {

    public static void main(String [] args){
        AWSCredentials credentials = null;
        try {
            credentials = new PropertiesFileCredentialsProvider("keys.props").getCredentials();
        } catch (Exception e) {
            throw new AmazonClientException(
                    "Cannot load the credentials from the credential profiles file. " , e);
        }

        TransferManager xfer_mgr = TransferManagerBuilder.standard().withS3Client(AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(credentials)).withRegion("us-west-2").build()).build();//TransferManagerBuilder.standard().build();

        try {
            MultipleFileDownload xfer = xfer_mgr.downloadDirectory(
                    "bucketName", null, new File("/Users/admin/Desktop/downloadFolder"));
            XferMgrProgress.showTransferProgress(xfer);
            XferMgrProgress.waitForCompletion(xfer);
        } catch (AmazonServiceException e) {
            System.err.println(e.getErrorMessage());
            System.exit(1);
        }
    }
}

Upvotes: 3

velika12
velika12

Reputation: 151

You can use downloadDirectory method from TransferManager class:

TransferManager transferManager = new TransferManager(new DefaultAWSCredentialsProviderChain());
File dir = new File("destDir");

MultipleFileDownload download =  transferManager.downloadDirectory("myBucket", "myKey", dir);
download.waitForCompletion();

As it is written in the documentation, this method:

Downloads all objects in the virtual directory designated by the keyPrefix given to the destination directory given. All virtual subdirectories will be downloaded recursively.

Upvotes: 9

Tiziano
Tiziano

Reputation: 346

Yes, use TransferManager.downloadFolder :)

Upvotes: -1

Max
Max

Reputation: 8836

You have to call the ListBucket API to get the list of files, then download each one individually with GetObject

Upvotes: 1

Related Questions