CODEWITHSUNDEEP
javaamazon-web-servicesamazon-s3aws-cli
Chris Maggiulli
Chris Maggiulli

Reputation: 3814

Downloading a directory from S3 works with Java SDK but not AWS CLI

Background

I need to download a directory from s3 using the AWS Command Line Interface. Whenever I attempt to download a file using the CLI I get an access denied error. However it downloads fine using the Java SDK. I cannot view the permissions associated with the key so I have to troubleshoot this as a black box. I included the code samples below

AWS CLI

I am using the following command in PowerShell to download the file through the AWS CLI

aws s3 cp s3://ec-sis-integration-test/fiscal/ .\

This command throws the following error

download failed: s3://ec-sis-integration-test/fiscal/ to ./ [WinError 5] Access is denied: 'C:\users\cmaggiul\Desktop\'

Java SDK

To troubleshoot this I created a sample Java application that attempts to download a directory from s3. The code is included below

AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withRegion(Regions.US_EAST_1).build();
TransferManager transferManager = TransferManagerBuilder.standard().withS3Client(s3Client).build();

File desktop = new File(System.getProperty("user.home"), "/Desktop"); 

MultipleFileDownload download =  transferManager.downloadDirectory(bucket, directory, desktop);
download.waitForCompletion();

This code successfully downloads the directory using the sample AWS_PROFILE

Troubleshooting steps

I have tried the following solutions

  1. Attempted to download empty directories and directories that are not empty
  2. Tried to add the --recursive flag to the CLI command
  3. Tried to change the ACL on files within the directory ( when I was attempting to download a non-empty directory )
  4. Tried to create the directory with the same AWS_PROFIlE I am using to download and with a different AWS_PROFILE
  5. I set up a docker image, download and installed the aws cli and set up the environment variables. I then ran

sudo aws s3 cp s3://ec-sis-integration-test/fiscal/ ./

And received the following error

download failed: s3://ec-sis-integration-test/fiscal/ to ./ [Errno 39] Directory not empty: '/root/aws-test/.9AA88F60' -> '/root/aws-test/'

But it works fine if I try and download a specific file

aws s3 cp s3://ec-sis-integration-test/fiscal/collections/COLL_20201228_1-EXP ./

It also works fine in PowerShell if I download a specific file

Question

Why can I download the file using the same AWS_PROFILE using the Java SDK but not the CLI?

Upvotes: 0

Views: 2297

Answers (1)

alexis-donoghue
alexis-donoghue

Reputation: 3387

 [WinError 5] Access is denied: 'C:\users\cmaggiul\Desktop\

This is unrelated to AWS. Your script doesn't have permissions to write to that directory.

Upvotes: 1

Related Questions