CODEWITHSUNDEEP
amazon-web-servicesamazon-s3boto3boto
JBernardo
JBernardo

Reputation: 33397

S3 pre signed url without path

I need a to have a way of letting a client upload data to S3 without showing them the full location (path) of the file. Is that something doable with AWS S3 pre-signed URL?

I'm using boto3 as such

s3.client.generate_presigned_url(
  ClientMethod='put_object', 
  ExpiresIn=7200,
  Params={'Bucket': BUCKET, 'Key': name}
)

But the outcome will be:

https://s3.amazonaws.com/MY_BUCKET/upload/xxxx-xxxx/file-name.bin?AWSAccessKeyId=XXXX&Signature=XXXX&Expires=XXXX

I need something like that won't show the Key name in the path (/upload/xxxx-xxxx/file-name.bin).

What other solutions do I have if not the pre-signed url?

Upvotes: 3

Views: 4862

Answers (2)

Matthew Pope
Matthew Pope

Reputation: 7669

This isn’t a complete answer, but it’s too long to be a comment.

I think you should be able to use API Gateway as a proxy for S3 to hide the path. You can still use pre-signed urls, but you might need to create pre-signed API Gateway urls rather than pre-signed S3 urls. I’ve never done it myself—nor will I be able to try it out in near future—but I’ll do my best to lay out how I think it’s done, and maybe someone else can try it and write up a more complete answer.

First, we need to set up an API gateway endpoint that will act as a proxy to S3.

AWS has a very thorough write-up on how to make a general proxy for S3, and I think you can make your custom endpoint point to a specific bucket and folder in S3 by modifying the PathOverride of the proxy. If you look at the screenshot of the PathOverrides in this section of the AWS documentation, you can see they have set the path override to {bucket}/{object}, but I think you could set the PathOverride to mySecretBucket/my/secret/folder/{object}, and then update the path mappings appropriately.

Next, you need to be able to use pre-signed urls with this proxy. There’s two ways you might be able to do this.

The first thing that might work is making the url signature pass through API Gateway to S3. I know it’s possible to map query parameters in a similar way to path parameters. You may need to perform some url encoding on the pre-signed URL’s signature param to make it work—I’m not entirely sure.

The other option is to allow Api Gateway to always write to S3, and require a signed request for calling your proxy endpoint. This SO question has a pretty detailed answer that looks to me like it should work.

Again, I know this isn’t a complete answer, and I haven’t tried to verify that this works, but hopefully someone can start with this and get to a complete answer for your question.

Upvotes: 1

grgizem
grgizem

Reputation: 310

I believe best way is distributing files with AWS Cloudfront. You can set the origin of the Cloudfront distribution to MY_BUCKET.s3.amazonaws.com. It is also possible to use subfolders like MY_BUCKET.s3.amazonaws.com/upload as origin.

Cloudfront will serve your files within S3 origin with generated CDN endpoint domain or it is possible to set and use custom domain as well.

https://d111111abcdef8.cloudfront.net/upload/xxxx-xxxx/file-name.bin https://uploads.example.com/upload/xxxx-xxxx/file-name.bin

if you use subfolder as origin:

https://uploads.example.com/xxxx-xxxx/file-name.bin

More info on setting S3 Bucket as origin on Cloudfront: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/DownloadDistS3AndCustomOrigins.html#concept_S3Origin

More info on using directory paths of S3 Bucket as origin: https://aws.amazon.com/about-aws/whats-new/2014/12/16/amazon-cloudfront-now-allows-directory-path-as-origin-name/

More info on Custom URLs: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/CNAMEs.html

Upvotes: 4

Related Questions