CODEWITHSUNDEEP
google-cloud-storage
scipio
scipio

Reputation: 45

Google Cloud Services put Fails when Using Signed URL

I am not able to PUT a file to google cloud services via a signed URL. When I try to do a PUT from a JS Client, I get:

"SignatureDoesNotMatch...The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method."

When I try to post the file using CURL, I get the same error.

The curl command I use is: 

#!/bin/bash 
URL="https://storage.googleapis.com/..."
echo $URL
curl $URL -H "Content-Type: image/jpg" --upload-file b.jpg

I have configured the bucket I intend to post data to based on the documentation, I have generated a service account with the key, and this key is used to generate the signed url.

The request I sign is of the form:

PUT

image/jpg
1234567890
my-bucket/b.jpg

where the expiration and bucket names are set and computed.

I have the following Groovy code to generate signed urls:

 public String sign(PrivateKey key, String toSign) {
    Signature signer = Signature.getInstance("SHA256withRSA");
    signer.initSign(key);
    signer.update(toSign.getBytes("UTF-8"));
    byte[] rawSignature = signer.sign();
    String s = new String(Base64.encodeBase64(rawSignature), "UTF-8");

    return s;
}

public String signUrl(PrivateKey key, String clientId, String method, String md5, String contentType,
                             long expiration, String gcsPath) {

    String toSign = "${method}\n${md5}\n${contentType}\n${expiration}\n${gcsPath}";
    String signature = sign(key, toSign);
    String url = java.net.URLEncoder.encode(signature);
    return url;
}

public String generateSignedUrl(PrivateKey key, String clientId, String method, String md5, String contentType,
                                       long expiration, String gcsPath) {

    String canonicalizedResource = "/${gcsPath}";
    String signature = signUrl(key, clientId, method, md5, contentType, expiration, canonicalizedResource);
    String finalUrl = "https://storage.googleapis.com/${gcsPath}?GoogleAccessId=${clientId}&Expires=${expiration}&Signature=${signature}"

    finalUrl
}

This code is accompanied with the following passing unit test lifted straight out of the gsutils github project (https://github.com/GoogleCloudPlatform/gsutil/blob/master/gslib/tests/test_signurl.py):

    @Test
void thatWeCanSignAPutUrlCorrectly() {
    String expected = """https://storage.googleapis.com/test/[email protected]&Expires=1391816302&Signature=A6QbgTA8cXZCtjy2xCr401bdi0e7zChTBQ6BX61L7AfytTGEQDMD%2BbvOQKjX7%2FsEh77cmzcSxOEKqTLUDbbkPgPqW3j8sGPSRX9VM58bgj1vt9yU8cRKoegFHXAqsATx2G5rc%2FvEliFp9UWMfVj5TaukqlBAVuzZWlyx0aQa9tCKXRtC9YcxORxG41RfiowA2kd8XBTQt4M9XTzpVyr5rVMzfr2LvtGf9UAJvlt8p6T6nThl2vy9%2FwBoPcMFaOWQcGTagwjyKWDcI1vQPIFQLGftAcv3QnGZxZTtg8pZW%2FIxRJrBhfFfcAc62hDKyaU2YssSMy%2FjUJynWx3TIiJjhg%3D%3D""";

    long expiration = 1391816302;

    String signedUrl = gsUtils.generateSignedUrl(privateKey, "[email protected]","PUT", "", "", expiration, "test/test.txt")

    assertEquals(expected, signedUrl);
}

Thank you for whatever insights you may be able to provide, I have been at this problem for a while.

Upvotes: 4

Views: 3632

Answers (2)

Adeyinka Adegbenro
Adeyinka Adegbenro

Reputation: 23

One reason that might cause this error (happened to me before) is when you generate a base64 encoded signature with your signed string, the encoded signature may contain illegal url characters + and /. Make sure you replace them in the string with %2B and %2F, respectively.

Upvotes: 1

Brandon Yarbrough
Brandon Yarbrough

Reputation: 38399

Debugging signed URL logic is difficult. There is a useful trick that helps, though. An error response like the one you describe will look like this:

<?xml version='1.0' encoding='UTF-8'?><Error><Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.</Message>
<StringToSign>PUT

text/jpeg
1472720161
/myBucket/test.txt</StringToSign></Error>

That last bit, <StringToSign>, is critical. The string in there is exactly the string that GCS will sign, and it's the string that you should also be signing. Compare you string against this one; it will probably be different in some way.

Also, because implementing this signing logic is tricky, the gcloud-java library has a signUrl() method that I recommend you use instead of implementing the logic yourself.

Upvotes: 4

Related Questions