CODEWITHSUNDEEP
javaamazon-web-servicesamazon-s3aws-java-sdk
Rohit Chouhan
Rohit Chouhan

Reputation: 261

How to download a file from s3 using provided url?

In my application I will get the url of s3 file like : https://s3.amazonaws.com/account-update/input.csv I have to download it and then process it. What I already done :

AmazonS3 s3 = new AmazonS3Client(credentials);
S3Object s3object = s3.getObject(new GetObjectRequest(
bucketName, key));

I am able to download the file by providing bucket name and key, but how can I download the file using the url(https://s3.amazonaws.com/account-update/input.csv) only?

Upvotes: 26

Views: 138177

Answers (8)

Riya John
Riya John

Reputation: 532

Just enter the url on the browser but make sure to replace \u0026 with & if you have downloaded the url via curl else you will get this error

<Error>
    <Code>AuthorizationQueryParametersError</Code>
    <Message>X-Amz-Algorithm only supports "AWS4-HMAC-SHA256"</Message>
</Error>

Upvotes: 1

Alberto Cerqueira
Alberto Cerqueira

Reputation: 1419

You can't, but you can make the file attachment in the upload.

For example:

ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.setContentType("application/csv;charset=utf-8");
objectMetadata.setContentDisposition("attachment");
PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, arquivo, file, objectMetadata).withCannedAcl(CannedAccessControlList.PublicRead);
amazonS3.putObject(putObjectRequest);

I hope it helps.

Upvotes: 0

Rez Moss
Rez Moss

Reputation: 4600

The best way is using the pre-signed S3 URL to achieve your needs. You can add expiration time to your signed URLs and after that the URL not available.

For more information read the following page:

https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURLJavaSDK.html

Upvotes: 1

tooptoop4
tooptoop4

Reputation: 330

using cli: aws s3 cp s3://bucket/prefix/key targetlocalfolder

Upvotes: -6

Jawad
Jawad

Reputation: 401

You may consider using the AWS SDK class AmazonS3URI as shown below:

URI fileToBeDownloaded = new URI(" https://s3.amazonaws.com/account-update/input.csv"); 

AmazonS3URI s3URI = new AmazonS3URI(fileToBeDownloaded);

S3Object s3Object = s3Client.getObject(s3URI.getBucket(), s3URI.getKey());

From here on, you should be able to utilise the s3Object obtained in a similar fashion to the s3Object shown in your code.

For more Java related AWS SDK examples on using this class check here

Upvotes: 23

Chris
Chris

Reputation: 3657

John Rotenstein is correct, you can download the file via the URL using standard curl/wget.

If you wanted to do this using Java, something like the following should do the trick; making use of the Apache HttpComponents package

private void downloadRequest(String url, String savedFile) {
    HttpClient client = HttpClients.createDefault();

    HttpGet httpGet = new HttpGet(url);
    HttpResponse response;
    try {
        response = client.execute(httpGet);
        HttpEntity entity = response.getEntity();
        if (entity != null) {
            try (FileOutputStream outstream = new FileOutputStream(savedFile)) {
                entity.writeTo(outstream);
            } catch (IOException e) {
                LOGGER.info(e.getMessage());
            }
        }

    } catch (IOException e) {
        e.printStackTrace();
    }
}

Upvotes: 0

John Rotenstein
John Rotenstein

Reputation: 270224

You could download the file via a standard curl/wget, the same as you would download any other file off the Internet.

The important part, however, is to enable access to the object from Amazon S3. A few options:

  • Make the object publicly-readable: This can be done via the console or CLI/API. However, anyone with that URL will be able to download it.
  • Create an Amazon S3 Bucket Policy that grants read access for the desired file/directory/bucket. But, again, anyone with the URL will be able to access those objects.
  • Keep the object private, but use a pre-signed URL that adds parameters to the URL to prove that you are permitted to download the object. This pre-signed URL is time-limited and can be generated with a few lines of code using current AWS credentials.

Upvotes: 22

ITAdminNC
ITAdminNC

Reputation: 219

To enable access by HTTP, you must set the bucket up as a Static Website in the S3 console. Be warned, this will expose all of your data to the web unless you set up proper S3 access controls.

The method you are accessing via the Java SDK will not use this type of connection, though. It will connect via the built-in S3 protocol. You should inspect your IAM Role or Policy to ensure you have the correct permissions (s3:GetObject). You will also need s3:ListBucket to see a 404 if the object does not exist.

Upvotes: 3

Related Questions