CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-sdkpre-signed-urlaws-sdk-js
bebbi
bebbi

Reputation: 2539

How to constrain client to send correct sha256 as a file key on s3 upload? (presigned url)

I need to create a signed url to upload a file to an s3 bucket. The s3 file key should be its sha256 hash.

The question then is: how can I make sure the client sends a valid hash? I'm creating the signed url at my lambda function and avoid passing the file through it, so the lambda of course cannot calculate the hash.

I'm thinking I can achieve this using 2 steps:

  1. Force the client to send its calculated sha256 with the upload. Based on spec I am assuming this will be auto-checked when providing it in a x-amz-content-sha256 header.

  2. Force client to send the same hash to the lambda so I can force it to be the key.

First, I tried this:

s3.getSignedUrl('putObject', { Key: userProvidedSha256 }, callback)

I tried adding a condition like { header: { 'X-Amz-Content-Sha256': userProvidedSha256 } }.

But I found no way of adding such a definition so that it actually forces the client to send a X-Amz-Content-Sha256 header.

Also, I would have taken the same approach to enforce a fixed required Content-Length header (client sends desired length to back-end, there we sign it), but not sure that would work because of this issue.

Because I found out that s3.createPresignedPost also lets me limit max attachment size and appears more flexible, I went down that route:

const signPostFile = () => {
  const params = {
    Fields: {
      key: userProvidedSha256
    },
    Expires: 86400,
    Conditions: [
      ['content-length-range', 0, 10000000],
      { 'X-Amz-Content-Sha256': userProvidedSha256]
    ]
  }

  s3.createPresignedPost(params, callback)
}

But while that works (it forces the client to send the enforced sha256 header, and the header gets passed, see request log below), it looks like the client now has to add the x-amz-content-sha256 into the form fields rather than the header. This seems to be as intended, but it clearly appears that s3 won't check the submitted file against the provided sha256: any file I append to the form is successfully uploaded even if the sha256 is a mismatch.

Any suggestion what's wrong, or how else I can enforce the sha256 condition, while also limiting content length?

Update: I'm using signature v4, and I've tried a S3 policy Deny for this condition:

Condition:
  StringEquals:
    s3:x-amz-content-sha256: UNSIGNED-PAYLOAD

Relevant request log for submitting a file containing the string "hello world":


----------------------------986452911605138616518063
Content-Disposition: form-data; name="X-Amz-Content-Sha256"

b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
----------------------------986452911605138616518063
Content-Disposition: form-data; name="key"

b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

Upvotes: 3

Views: 3203

Answers (1)

matsev
matsev

Reputation: 33779

To my knowledge S3 does not provide sha256 by default. However, by listening to S3 events you can implement a Lambda function that does this automatically for you. Here is a suggestion that comes to mind:

  1. The client requests a S3 signed url based on the user provided sha256
  2. The client uploads the file using the signed url
  3. A Lambda function is configured to listen to s3:ObjectCreated:* events from the upload bucket
  4. When the upload is completed, the Lambda function is triggered by a S3 Message Event. A part of the event is the S3 object key
  5. The Lambda function downloads the uploaded file and re-calculate the sha256
  6. The Lambda function deletes the file if the calculated sha256 value differs from the sha256 value that was provided by the client (either as the object key or available from the objects metadata)

Alternatively, if the main objective is to verify the file integrity of uploaded files, S3 provides another option to use sha256 when calculating checksums.

  1. Configure a bucket policy to only accept requests that have been signed

  2. Configure the client AWS S3 sdk to use AWS signature version 4, e.g.

    const s3 = new AWS.S3({apiVersion: '2006-03-01', signatureVersion: 'v4'});

  3. The S3.putObject() function will sign the request before uploading the file

S3 will not store an object if the signature is wrong as described in the AWS CLI S3 FAQ

Upvotes: 1

Related Questions