CODEWITHSUNDEEP
javascriptnode.jsamazon-s3data-transfertemporal-workflow
sooraj
sooraj

Reputation: 1

Syncing Data Transfer Between S3 Buckets Using AWS SDK and Temporal SDK"

able to transfer data from s3 source bucket to s3 destination bucket and its work in sync way like when ever the source get update i should send automatically that date to the destination bucket and if any error happens in the get or in the post it do the process until it finish the task so I want to complete this without use of any AWS paid service and "temporal sdk" I am using it if there is any other way please help to solve this problem preferred programming language is nodejs

temporal configuration code with setup explanation or any other technology which can able to solve the problem with nodejs

Upvotes: -2

Views: 185

Answers (2)

repsick3r
repsick3r

Reputation: 31

For this use case we can use temporal to orchestrate this duplication in the bucket B.

The service which calls to update in bucket A, should have two activities.

Activity A : Stores in bucket A and returns.

Activity B : Stores in bucket B and returns.

Run a temporal workflow with these two activites sequentially with enough number of retries and timeouts.

Although as John was mentioning you could use AWS replication, so that you can offload all this stuff.

Upvotes: 0

John Rotenstein
John Rotenstein

Reputation: 270089

It appears that your requirement is:

  • Whenever an object is created in Bucket-A
  • The object should be automatically copied to Bucket-B

You can do this with Amazon S3 Replication

There are two options available:

  • Same Region Replication, and
  • Cross Region Replication

There is no extra charge for using this feature -- you just pay for the normal cost of copying and storing the additional objects.

Upvotes: 0

Related Questions