CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-ec2
Parijat Bose
Parijat Bose

Reputation: 390

AWS s3 mv command deletes the source directory

The objective is to move all the files from s3 "dir2" directory to EMR directory "mydir".

I am using the command:

aws s3 mv s3:///dir1/dir2/ /mnt/mydir/ --recursive

This command gets executed but the dir2 directory from s3 gets deleted. The files within dir2 although moves to mydir of EMR.

How can I only move the files from source dir of s3 without removing the source directory?

Upvotes: 0

Views: 3934

Answers (2)

chris
chris

Reputation: 37460

When dealing with multiple objects, you want to use sync not cp or mv:

aws s3 sync s3:///dir1/dir2/ /mnt/mydir/

There are ways to load data into EMR directly from S3, so you may want to look into those.

Update: I have confirmed that:

aws s3 mv s3://bucket/f1/f2 . --recursive

Will move all of the files inside f2/ while leaving f2 in the bucket.

Upvotes: 2

Mark B
Mark B

Reputation: 200562

Directories or folders do not actually exist in S3. What you are calling a directory is simply a common file name prefix in S3. When there are no files with that prefix anymore then the "directory" does not exist anymore in S3.

Upvotes: 2

Related Questions