Google Cloud Storage get signedUrl from CDN npm

Question

I am using a code as the following to create a signed Url for my content:

var storage = require('@google-cloud/storage')();
var myBucket = storage.bucket('my-bucket');
var file = myBucket.file('my-file');

//-
// Generate a URL that allows temporary access to download your file.
//-
var request = require('request');

var config = {
  action: 'read',
  expires: '03-17-2025'
};

file.getSignedUrl(config, function(err, url) {
  if (err) {
    console.error(err);
    return;
  }

  // The file is now available to read from the URL.

});

This creates an Url that starts with https://storage.googleapis.com/my-bucket/

If I place that URL in the browser, it is readable.

However, i guess that URL is a direct access to the bucket file and is not passing through my configured CDN.

I see that in the docs (https://cloud.google.com/nodejs/docs/reference/storage/1.6.x/File#getSignedUrl) you can pass a cname option, which transforms the url to replace https://storage.googleapis.com/my-bucket/ to my bucket CDN.

HOWEVER when I copy the resulting URL, the sevice account or resulting url doesn't seem to have access to the resource.

I have added the firebase admin service account to the bucket but still I get no access.

Also, from the docs, the CDN signed url seems a lot different from the one signed through that API. Is it possible to create from the api a CDN signed url, or should i manually create it as explained in: https://cloud.google.com/cdn/docs/using-signed-urls?hl=en_US&_ga=2.131493069.-352689337.1519430995#configuring_google_cloud_storage_permissions?

Answer 1

Modified @htafoya code slightly,

async generateSignedUrlForCdnResource(fileName: string): Promise<string> {
    const url =
      this.configService.get<string>(GCP_BACKEND_BUCKET_BASE_URL) + fileName;
    const base64SignedKey = this.configService.get<string>(
      GCP_SIGNING_BASE_64_ENCODED_KEY,
    );
    const expirationTime = Date.now() + 2 * 60; // 2 minutes
    const strippedUrl = url.trim();

    const parsedUrl = new URL(strippedUrl);
    const queryParams = new URLSearchParams(parsedUrl.search);

    const expirationTimestamp = Math.floor(expirationTime);

    const decodedKey = Buffer.from(base64SignedKey, 'base64url');

    const urlToSign = `${strippedUrl}${queryParams.toString() ? '&' : '?'}Expires=${expirationTimestamp}&KeyName=${this.configService.get<string>(GCP_SIGNING_KEY_FILE_NAME)}`;

    const hmac = crypto.createHmac('sha1', decodedKey);
    hmac.update(urlToSign);
    const digest = hmac.digest();
    const signature = Buffer.from(digest).toString('base64url');

    return `${urlToSign}&Signature=${signature}`;
  }

Answer 2

From what Google documents here. The answer provided by @htafoya seem legit. However, I spent a couple of hours to struggle why the signed URL not working as CDN endpoint complains access denied. Eventually I found the code using crypto module doesn't produce the same hmac-sha1 hash value as what gcloud compute sign-url computed, I still don't know why.

At the same time, I see this lib (jsSHA) is pretty cool, it generates the HMAC-SHA1 hash value exactly the same as gcloud and it has a neat API, so I think I should comment here so that if others have the same struggle will benefit from this, this is the final code I used to sign gcloud cdn URL:

    import jsSHA from 'jssha';
    const url = `https://{domain}/{path}`;
    const expire = Math.round(new Date().getTime() / 1000) + daySeconds;
  const extendedUrl = `${url}${url.indexOf('?') > -1 ? "&" : "?"}Expires=${expire}&KeyName=${keyName}`;

  // use jssha
  const shaObj = new jsSHA("SHA-1", "TEXT", { hmacKey: { value: signKey, format: "B64" } });
  shaObj.update(extendedUrl);
  const signature = safeSign(shaObj.getHMAC('B64'));
  return `${extendedUrl}&Signature=${signature}`;

working great!

Answer 3

The Cloud CDN content delivery network works with HTTP(S) load balancing to deliver content to your users. Are you using HTTPS Load Balancer to deliver content to your users? You can see this attached document[1] on using Google Cloud CDN and HTTP(S) load balancing and inserting content into the cache.

[1] https://cloud.google.com/cdn/docs/overview [2] https://cloud.google.com/cdn/docs/concepts

What error code are you getting? Can you use the curl command and send the output with the error code for further analysis.

Could you confirm that configuration you have done meets the requirement of cacheability, as not all the HTTP response are cacheable? Google Cloud CDN caches only those responses that satisfy specific conditions [3], please confirm. Upon confirmation, I will do further investigation and advise you accordingly.

[3] Cacheability: https://cloud.google.com/cdn/docs/caching#cacheability

Could you provide me the output of this two command below, which will help me to verify if there is a permission issue on these objects? These commands will dump all the current permission settings on the object.

gsutil acl get gs://[full_path_to_file_to_be_cached] gsutil ls -L gs://[full_path_to_file_to_be_cached]

For more details on permission, refer to this GCP documentation [4]

[4] Setting bucket permissions: https://cloud.google.com/storage/docs/cloud-console#_bucketpermission

No, it is not possible to create from the API a CDN signed URL

Answer 4

For anyone interested in the node code for that signing:

    var url = 'URL of the endpoint served by Cloud CDN';
    var key_name = 'Name of the signing key added to the Google Cloud Storage bucket or service';
    var key = 'Signing key as urlsafe base64 encoded string';
    var expiration = Math.round(new Date().getTime()/1000) + 600; //ten minutes after, in seconds

    var crypto = require("crypto");
    var URLSafeBase64 = require('urlsafe-base64');

    // Decode the URL safe base64 encode key
    var decoded_key = URLSafeBase64.decode(key);

    // buILD URL
    var urlToSign = url 
            + (url.indexOf('?') > -1 ? "&" : "?")
            + "Expires=" + expiration
            + "&KeyName=" + key_name;

    //Sign the url using the key and url safe base64 encode the signature
    var hmac = crypto.createHmac('sha1', decoded_key); 
    var signature = hmac.update(urlToSign).digest();
    var encoded_signature = URLSafeBase64.encode(signature);

    //Concatenate the URL and encoded signature
    urlToSign += "&Signature=" + encoded_signature;