CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-emramazon-kms
Sanket_patil
Sanket_patil

Reputation: 300

How to encrypt data in S3 from EMR without the need to mention sse and key-id in request header?

I have created up an EMR cluster with Amazon S3 SSE-S3 Enabled.

Also I configured

fs.s3.enableServerSideEncryption = true
fs.s3.serverSideEncryption.kms.keyId = key-id

in emrfs-site.xml

I am writing my data as follows to s3 

aws s3 cp /home/hadoop/test.csv s3://my-bucket/enc/ --sse aws:kms /key-id-     here-/

This Will save my data encrypted with kms, But I dont want to send any parameters for encryption and whatever data sent from current EMR cluster should be encrypted.Just by 

aws s3 cp /home/hadoop/test.csv s3://my-bucket/enc/

Any way how I can do it ?

Upvotes: 0

Views: 1365

Answers (2)

Ajak6
Ajak6

Reputation: 747

Try enabling default bucket encryption and specified the key you want to use to encrypt all objects in the bucket. Post that all data written should be encrypted by default with your keys. I tested writing a file using boto in an encrypted bucket and that object does get encrypted with bucket key. I am not sure what is different with EMR

https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-bucket-encryption.html

Upvotes: 0

Michael - sqlbot
Michael - sqlbot

Reputation: 179124

No, it doesn't work that way.

If you want your target object encrypted, you will need to provide appropriate encryption related request headers

http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html

S3 decrypts copied objects for copying, then re-encrypts them at the destination, and this appears to be true even if the keys are the same.

Upvotes: 1

Related Questions