CODEWITHSUNDEEP
javascalaamazon-web-servicesamazon-s3
Pasha Bitz
Pasha Bitz

Reputation: 787

SignatureDoesNotMatch in AWS Java SDK for S3

I have a bucket belonging to another account. Using the AWS CLI I am able to access (list and read objects) this bucket. For example:

aws s3 ls s3://somebucket/foo/bar

Lists the objects.

Trying to recreate the same using the Java SDK (in Scala) I am getting the above exception (SignatureDoesNotMatch).

Here is the code:

package com.myco.sample

class TestCase() {
    val credentials = new com.amazonaws.auth.BasicAWSCredentials(
        "ACCESS_KEY_ID", 
        "SECRET_ACCESS_KEY"
    )
    val s3 = new com.amazonaws.services.s3.AmazonS3Client(credentials)
    val endpoint = "somebucket.s3-us-west-2.amazonaws.com"
    s3.setEndpoint(endpoint)

    try {
        val objs = s3.listObjects("foo/bar")
    } catch {
        case ace: com.amazonaws.services.s3.model.AmazonS3Exception => {
            println(ace.getAdditionalDetails)
        }
    }
}

The call to listObjects throws an exception. The output is:

com.amazonaws.services.s3.model.AmazonS3Exception: The request signature we calculated does not match the signature you provided. Check your key and signing method. (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: XXXXXXXXX), S3 Extended Request ID: XXXXXXXXXXXXXXXXXXX=
{SignatureProvided=XXXXXXXXXXXXX=, StringToSign=Wed, 06 Jan 2016 04:32:38 GMT
/somebucket/foo/bar/, AWSAccessKeyId=XXXXXX, Error=XXXXXXXXXXXX=, StringToSignBytes=XXXXXXXXX}

When not providing the endpoint as above, I am getting a different error: The bucket you are attempting to access must be addressed using the specified endpoint

After setting the endpoint, I've tried multiple ways of passing the "bucket" parameter to listObjects, all of which didn't work.

Not sure why the signature that's being generated behind the scenes is incorrect. Any ideas?

Upvotes: 5

Views: 10291

Answers (2)

Vasyl Sarzhynskyi
Vasyl Sarzhynskyi

Reputation: 3955

In my case, SignatureDoesNotMatch error occurred after upgraded maven dependencies without changes in my code (so credentials are correct and were not changed). After upgrading dependency org.apache.httpcomponents:httpclient from version 4.5.6 to 4.5.7 (actually it was upgrade of Spring Boot from 2.1.2 to 2.1.3, and there bom has specified httpclient version), code became throw exceptions while doing some AWS SDK S3 requests like AmazonS3.getObject.

After digging into the root cause, I found that httpclient library did breaking changes with normalized URI, that affected Java AWS SDK S3. Please take a look for opened GitHub ticket org.apache.httpcomponents:httpclient:4.5.7 breaks fetching S3 objects for more details.

Upvotes: 3

ataylor
ataylor

Reputation: 66109

That error typically means the credentials aren't correct.

val credentials = new com.amazonaws.auth.BasicAWSCredentials(
    "ACCESS_KEY_ID", 
    "SECRET_ACCESS_KEY"
)

Are you using the actual access key and secret key in your code? Do they match the values in your ~/.aws/credentials file?

You can try creating the AmazonS3Client without explicitly providing credentials using the default constructor. The default behavior is to use the values in ~/.aws/credentials just like the CLI.

To rule out credentials issues, you can turn on logging in the CLI and compare it with the SDK logs. Try:

aws --debug s3 ls s3://somebucket/foo/bar

You should see something like this:

...
2016-01-06 13:29:01,306 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2016-01-06 13:29:01,306 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2016-01-06 13:29:01,306 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2016-01-06 13:29:01,306 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
...

Next, enable SDK logging as documented here: http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-logging.html. You should just need to provide the log4j jar and the example log4j.properties file.

Here you should see this:

...
2016-01-06 13:26:47,621 [main] DEBUG com.amazonaws.auth.AWSCredentialsProviderChain -  Unable to load credentials from EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
2016-01-06 13:26:47,621 [main] DEBUG com.amazonaws.auth.AWSCredentialsProviderChain -  Unable to load credentials from SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey)
2016-01-06 13:26:47,636 [main] DEBUG com.amazonaws.auth.AWSCredentialsProviderChain -  Loading credentials from com.amazonaws.auth.profile.ProfileCredentialsProvider@42561fba
...

If that turns out not to be the issue, you can examine the logs in detail to further diagnose the problem.

Upvotes: 2

Related Questions