CODEWITHSUNDEEP
amazon-web-servicesamazon-s3google-cloud-platformgoogle-cloud-storage
user3325789
user3325789

Reputation: 413

gs:// s3:// http:// equivalent

I see URIs for google cloud storage with the gs:// scheme and URIs for aws S3 with the s3:// scheme. How are these schemes translated into http and what do they mean?

How do these schemes know which region to point to and how are they expanded into full http bucket names?

Upvotes: 3

Views: 1278

Answers (2)

Brandon Yarbrough
Brandon Yarbrough

Reputation: 38379

gs:// is a way of referencing a particular GCS object used by the gsutil command-line utility and a small number of Google Cloud APIs to reference GCS objects. For example, a Cloud Vision API request might have a section like this:

   "source":{
      "imageUri": "gs://bucket_name/path_to_image_object"
    }

Or a gsutil command might look like this:

$> gsutil cp resume.pdf gs://my_bucket/

There is no single mapping between gs:// and https://. It is simply a way to specify a particular GCS object. An example HTTP mapping from a gs:// URI might be something like https://storage.googleapis.com/bucket_name/object_name.

Upvotes: 1

Tuxdude
Tuxdude

Reputation: 49473

Transparent or Opaque Identifiers

The URIs are just identifiers that can be understood only by the respective Cloud service providers (like Google and Amazon here).

The URIs could either be transparent (i.e. directly have the region name and other location details as part of the URI) or it might be opaque (i.e. not part of the URI) and the service internally has a lookup table of some sort to map each bucket name to the location where the data is stored.

Since the only way to access the content at these URIs is using the respective Cloud providers, you need not have to worry about that detail and instead care only about how/when to use that content.

APIs

If you need to know more information about a particular bucket hosted at any of these Cloud providers, you can use their APIs (and/or tools/libraries) to access the metadata about the bucket. This will also have a lot of metadata that you're looking for.

Analogy to email addresses

Think of this as your email address provided by your email service provider. The email address acts as a unique identifier to identify your account. The data (i.e. emails, contacts) corresponding to an email account might not be stored in one particular region. The service provider has the freedom to decide how they wish to store/shard the data for different email addresses. They could also be replicating the data at more than one region/country/continent for both higher availability as for redundancy. All these details can also change over time if they wish to redesign, but still remain transparent to you since you only access the account using the email address. Same is true with Cloud Storage providers as well.

Upvotes: 3

Related Questions