CODEWITHSUNDEEP
javascriptnode.jsamazon-web-servicesamazon-s3aws-sdk
Cherry
Cherry

Reputation: 33628

Is s3 sync supported from sdk instead of cli?

Here is answer about how move files from one s3 bucket to another using sync command from command line (cli):

aws s3 sync s3://from_my_bucket s3://to_my_other_bucket

Can this be invoked from javascript sdk? With searching I did not find any sync named method. So is it supported in sdk?

Upvotes: 15

Views: 12867

Answers (4)

GoForth
GoForth

Reputation: 656

This should be the defacto answer: No but there are official examples. Here is the README to their Go example: https://github.com/aws/aws-sdk-go/blob/main/example/service/s3/sync/README.md

Upvotes: 0

Jatin Verma
Jatin Verma

Reputation: 73

I found this really helpful article on cloning S3 bucket using Node.JS package "aws-sdk": [Node Clone S3 Bucket by Rajesh Babu][1]

I am quoting his approach here:

  • Fetch the list of Keys from the bucket and the target Prefix. (check AWS-SDK Javascript APIs)
  • Separate the files and directories, because we clone the directories and download the files.
  • Clone all the directories first, and then move on to download the files.
  • Download the files through streams and log success and failure respectively.
[1]: https://blog.bitsrc.io/a-practical-guide-to-building-a-node-js-service-on-an-aws-s3-bucket-aff19105ba83

Upvotes: 1

jfrumar
jfrumar

Reputation: 1980

I needed to sync thousands of files, and decided to make use of AWS Data Pipeline to do it. I update the pipeline definition to point to the correct source and destination folders from my Lambda function, and then invoke it.

It uses 2 S3DataNodes and a CopyActivity.

Upvotes: 0

Noel Llevares
Noel Llevares

Reputation: 16087

You can see how this node package does it.

Basically, it makes use of several S3 instance methods to sync two buckets.

Upvotes: 0

Related Questions