CODEWITHSUNDEEP
azureamazon-s3
Salam
Salam

Reputation: 11

Sync Data From Azure Blob Storage to AWS S3

How do I to keep a particular file in AWS in sync with the source file that is initially in my Azure blob storage?

Azure blob storage = Source

AWS = Target

I want to constantly keep both files in sync, by updating the file in AWS (Target) with the file in Azure blob storage (Source)

Upvotes: 1

Views: 2927

Answers (2)

Rene B.
Rene B.

Reputation: 7364

You can easily sync your data from Azure blob storage to s3 with RClone as described here:

https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/migrate-data-from-microsoft-azure-blob-to-amazon-s3-by-using-rclone.html

Example: copy

This command copies data from the source Azure Blob container to the destination S3 bucket.

rclone copy AZStorageAccount:blob-container s3:examplebucket-01

Example: sync

This command synchronizes data between the source Azure Blob container and the destination S3 bucket.

rclone sync AZStorageAccount:blob-container s3:examplebucket-01

Upvotes: 0

adp
adp

Reputation: 1271

You can use the open-source Node.js package azure-blob-to-s3 for the migration of the content from Azure Blob Storage to Amazon S3. An example CLI command is as follows:

azure-s3 \
  --concurrency 10 \
  --azure-connection "..." \
  --azure-container my-container \
  --aws-bucket my-bucket \
  --aws-prefix my-prefix

Github page of azure-s3: https://github.com/bendrucker/azure-blob-to-s3

Also, this blog page describes the use-case in detail: https://github.com/bendrucker/azure-blob-to-s3https://aws.amazon.com/blogs/storage/one-way-to-migrate-data-from-azure-blob-storage-to-amazon-s3/

Upvotes: 3

Related Questions