software.amazon.awssdk.services.s3.model.s3exception: null with using GetObjectRequest

Question

I am trying to use this code but i'm getting an exeption

var awsBasicCredentials  =  Packages.software.amazon.awssdk.auth.credentials.AwsBasicCredentials.create("ackey","secretkey");
var credentials =  Packages.software.amazon.awssdk.auth.credentials.StaticCredentialsProvider.create(awsBasicCredentials);

var region = Packages.software.amazon.awssdk.regions.Region.AWS_CN_GLOBAL;
var uri =  Packages.java.net.URI.create("http://host");
var client =  Packages.software.amazon.awssdk.services.s3.S3Client.builder()
                                                            
                                                            .credentialsProvider(credentials)
                                                            .region(region)
                                                            .endpointOverride(uri)
                                                            .build(); 
var request  = Packages.software.amazon.awssdk.services.s3.model.GetObjectRequest.builder()
                                                                        .bucket("/bucketname")                                      
                                                                        .key("key")
                                                                        .build();
var response = client.getObject(request);
return response;

I am using /bucketname because the final link looks like host/bucketname instead bucketname.host

S3Exeption:

Caused by: software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status Code: 403, Request ID: tx0000000000000162eece0-00633fea7f-306fc-msk-rt1)
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125)
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82)
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60)
    

Answer 1

You don't have the required permissions to perform a GetObject request against the specified S3 bucket.

I know the null in the Exception throws one off a bit, but if you look subsequent to that you'll see that the request returned a 403 status code. 403 status code means "Access Denied". So I would review your S3 bucket policy and make sure that your user have the required s3:GetObject permission required to perform a GetObject request. You can find some example bucket policies by searching the internet or looking through the AWS documentation, for example here or here.

It's also worthwhile reading the documentation on the GetObject request. From that documentation:

If the object that you request doesn’t exist, the error that Amazon S3 returns depends on whether you also have the s3:ListBucket permission.

If you have the s3:ListBucket permission on the bucket, Amazon S3 returns an HTTP status code 404 Not Found error.

If you don’t have the s3:ListBucket permission, Amazon S3 returns an HTTP status code 403 Access Denied error.

So the other reason you might be getting a 403 status code might be that the specified key doesn't exist in the bucket. And as per the preceding quote, the API will return a 403 status code in that case if you don't have the s3:ListBucket permission. Maybe it's a good practice to always consider assigning s3:ListBucket permission to users who have s3:GetObject permission to make their life a bit easier - to make it easier for them to differentiate between (1) a bucket permission issue and (2) a bad key.

Aside:

The link between your GetObject request and the s3:GetObject permission are clear. The link between the GetObject request and the s3:ListBucket permission is probably a bit more obscure. So yeah, if you ever encounter this problem again - and it's not just a case of "okay, I was trying to do something I don't have permission to do" - you might have to do some reading.

An example of what I'm alluding to: You can issue a S3 HeadBucket request to determine if a S3 bucket exists. It also requires the s3:ListBucket permission - so without it, you'll also get a 403 Access Denied response.

Interesting, that one link I shared earlier actually lists a few cases you might encounter. I'm just putting it here - might help others that stumble upon your post and are also encountering hard-to-debug 403 permission errors.

The following shows different types of mapping relationship between S3 API operations and the required policy actions.

One-to-one mapping with the same name. For example, to use the PutBucketPolicy API operation, the s3:PutBucketPolicy policy action is required.

One-to-one mapping with different names. For example, to use the ListObjectsV2 API operation, the s3:ListBucket policy action is required.

One-to-many mapping. For example, to use the HeadObject API operation, the s3:GetObject is required. Also, when you use S3 Object Lock and want to get an object's Legal Hold status or retention settings, the corresponding s3:GetObjectLegalHold or s3:GetObjectRetention policy actions are also required before you can use the HeadObject API operation.

Many-to-one mapping. For example, to use the ListObjectsV2 or HeadBucket API operations, the s3:ListBucket policy action is required.

Bucket policy example:

Okay, I actually just checked my own code and I do have a S3 bucket policy that gives s3:GetObject permission. It also blocks people from retrieving objects from the S3 bucket without "encryption in transit", eg HTTP. In this specific example I didn't give the user s3:ListBucket permission:

{
  "Version" : "2012-10-17",
  "Statement" : [ {
    "Sid" : "Only-allow-encrypted-traffic",
    "Effect" : "Deny",
    "Principal" : "*",
    "Action" : "s3:*",
    "Resource" : [ "arn:aws:s3:::YOUR_BUCKET_NAME", "arn:aws:s3:::YOUR_BUCKET_NAME/*" ],
    "Condition" : {
      "Bool" : {
        "aws:SecureTransport" : "false"
      }
    }
  }, {
    "Sid" : "Allow-public-access",
    "Effect" : "Allow",
    "Principal" : "*",
    "Action" : "s3:GetObject",
    "Resource" : "arn:aws:s3:::YOUR_BUCKET_NAME/*",
    "Condition" : {
      "Bool" : {
        "aws:SecureTransport" : "true"
      }
    }
  } ]
}

Answer 2

Bucket names can consist only of lowercase letters, numbers, dots (.), and hyphens (-).

Here is S3 Java code that works and returns a byte[] that represents the object that is located in the given Amazon S3 bucket.

In this example, the path represents the local file system where the object is written to . FOr example, a PDF file.

package com.example.s3;

// snippet-start:[s3.java2.getobjectdata.import]
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.core.ResponseBytes;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.awssdk.services.s3.model.GetObjectResponse;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
// snippet-end:[s3.java2.getobjectdata.import]

/**
 * Before running this Java V2 code example, set up your development environment, including your credentials.
 *
 * For more information, see the following documentation topic:
 *
 * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
 */

public class GetObjectData {

    public static void main(String[] args) {

     final String usage = "\n" +
         "Usage:\n" +
         "    <bucketName> <keyName> <path>\n\n" +
         "Where:\n" +
         "    bucketName - The Amazon S3 bucket name. \n\n"+
         "    keyName - The key name. \n\n"+
         "    path - The path where the file is written to. \n\n";

        if (args.length != 3) {
            System.out.println(usage);
            System.exit(1);
        }

        String bucketName = args[0];
        String keyName = args[1];
        String path = args[2];

        ProfileCredentialsProvider credentialsProvider = ProfileCredentialsProvider.create();
        Region region = Region.US_EAST_1;
        S3Client s3 = S3Client.builder()
            .region(region)
            .credentialsProvider(credentialsProvider)
            .build();

        getObjectBytes(s3,bucketName,keyName, path);
        s3.close();
    }

    // snippet-start:[s3.java2.getobjectdata.main]
    public static void getObjectBytes (S3Client s3, String bucketName, String keyName, String path) {

        try {
            GetObjectRequest objectRequest = GetObjectRequest
                .builder()
                .key(keyName)
                .bucket(bucketName)
                .build();

            ResponseBytes<GetObjectResponse> objectBytes = s3.getObjectAsBytes(objectRequest);
            byte[] data = objectBytes.asByteArray();

            // Write the data to a local file.
            File myFile = new File(path );
            OutputStream os = new FileOutputStream(myFile);
            os.write(data);
            System.out.println("Successfully obtained bytes from an S3 object");
            os.close();

        } catch (IOException ex) {
            ex.printStackTrace();
        } catch (S3Exception e) {
            System.err.println(e.awsErrorDetails().errorMessage());
            System.exit(1);
        }
    }
    // snippet-end:[s3.java2.getobjectdata.main]
}