CODEWITHSUNDEEP
amazon-s3s3cmd
ui_90jax
ui_90jax

Reputation: 759

S3 Sync Not Writable Warning

I am trying to use s3cmd sync (vanilla command remote to local), but for each file I get a not writable: Operation not permitted warning (see below). The files appear to download correctly, however, and if I do an ls on the file it is there. I tried using the --disable-multipart flag but this made no difference. I am using s3cmd version 1.5.0-rc1.

Is there any reason why I might be receiving these warnings?

WARNING: MD5 signatures do not match: computed=11ff15bf.., received="b119cb85b109c6.."
WARNING: ./path/part-r-00003.gz.parquet not writable: Operation not permitted
s3://bucket/path/part-r-00003.gz.parquet -> <fdopen>  [31 of 72] 36101458 of 36101458   100% in 2s    16.30 MB/s  done

Upvotes: 12

Views: 2153

Answers (1)

Stragulus
Stragulus

Reputation: 1113

A likely reason is that the S3 objects have been stored with filesystem attributes. When you sync them to a local directory, by default s3cmd sync will try to restore these attributes, including uid,gid ownership.

To prevent s3cmd from doing this, run it with the --no-preserve option, e.g.:

s3cmd sync --no-preserve s3://bucket/my/key/prefix .

Upvotes: 33

Related Questions