CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
Josh Beam
Josh Beam

Reputation: 19772

Can you request an object from S3 without knowing its extension?

Say I have a bucket called uploads with two directories, both of which contain images.

  1. The first directory, called catalog, has images with various extensions (.jpg, .png, etc.)

  2. The second directory, called brands, has images with no extensions.

I can request uploads/catalog/some-image.jpg and uploads/brands/extensionless-image, and they both return an image as I expect.

We're already using a third-party service, imgix, which is just an image-processing CDN that links to the S3 bucket so that we can request, say, a smaller or cropped version of the image in the bucket.

Ideally, I'd like to keep the images and objects in their current formats in the bucket, but I would like the client-side to be agnostic about which file it is requesting. In other words, I'd like to request some-image, and even though it may or may not actually have an extension in the bucket, I'd still like to somehow "intelligently guess" the image I'm requesting. We'll also assume that there are no collisions, i.e., there will never be an image some-image.jpg and some-image with both the same name (our objects are named with a collision-less algorithm).

This is what I've tried:

Anyone run into something similar before? Thoughts?

Upvotes: 1

Views: 3299

Answers (2)

Michael - sqlbot
Michael - sqlbot

Reputation: 178956

I suspect your best bet is going to be renaming the images. Not that there aren't other solutions, but because that is probably going to be the simplest and most straightforward approach.

First, S3 will not guess. The key on an S3 object is an opaque string from S3's perspective. The extension has no meaning, and even the slashes delimiting "directories" have no intrinsic meaning to S3. (Deleting a "directory" in S3 means sending a delete request for every individual object in the directory. The console creates a convenient illusion by doing this for you.)

S3 has redirect rules, but they only match and manipulate path prefixes, not suffixes, so no help there.

It would be possible, using a reverse proxy in front of S3, to inspect requests and for any 404 or 403, the proxy could retry the request with alternate extensions, until it found one that worked, and it could potentially "learn" the right extension for use on subsequent requests, but then you'd have the added turn-around time and additional cost for multiple requests.

I have developed systems whose job it is to "find" things requested over HTTP by trying multiple back-end URLs, without the requester being aware of the "hunting" going on in the background, and it can be very useful... but that is a much more complicated solution than you would probably want to consider, particularly in light of the fact that every millisecond counts when it comes to image loading.

There is no native solution for magic guessing with S3. You pretty much have to ask it for exactly what you want. Storage in S3 is cheap enough, of course, that you could probably duplicate your content, with and without extensions, without giving too much thought to the cost. If you used a Lambda event on the bucket, you could even automate the process of copying "kitten.jpg" to "kitten" each time "kitten.jpg" was modified.

Upvotes: 2

Julio Faerman
Julio Faerman

Reputation: 13501

If the content-type is set correctly in your object metadata, you should be fine regardless of extensions. If content-type header is not set, you can set it, for example using ImageMagick Identify to discover the image type and AWS CLI to set it.

Upvotes: 0

Related Questions