CODEWITHSUNDEEP
bashunixamazon-s3aws-cli
chickens
chickens

Reputation: 22354

aws s3 cli sync with watching folder live

What would be the bash command that will continuously watch jsondata folder and sync that folder with s3 when changes happen to local folder.

Now I am running this command but it does not have a -watch flag.

aws s3 sync jsondata s3://mybucket/jsondata/

Upvotes: 6

Views: 7137

Answers (4)

Julian Krispel-Samsel
Julian Krispel-Samsel

Reputation: 7734

I too had the same problem and came across this stackoverflow question and figured there has to be a more convenient way to do this.

So I wrote a little npm module exactly for this purpose. it watches a folder and uploads file changes to an s3 bucket automatically. It's called s3-auto-sync

Once installed you can just run s3-auto-sync -r us-east-1 -b my-bucket -d ./my-folder. That's it.

PS: Use a bucket in the same region as you are located, this will significantly improve upload speed.

Upvotes: 1

Valsaraj Viswanathan
Valsaraj Viswanathan

Reputation: 1567

S3 Sync is a bash script that will setup automatic syncing between your local directory and Amazon S3, utilizing the inotify linux kernel utility and s3cmd. It can also sync from your S3 bucket to a local directory via cron.

https://github.com/jasonrichardsmith/s3_sync

Upvotes: 2

Hassan Murtaza
Hassan Murtaza

Reputation: 1013

In Linux, you can use incrontab utility to watch over files and folders. Trigger the script whenever you receive a new file in folder. The script will contain aws sync command.

Upvotes: 3

Ashaman Kingpin
Ashaman Kingpin

Reputation: 1577

Seems like there are many options but here is one from a quick google search:

https://pypi.org/project/watchdog/

Python script that can register for filesystem notifications and take an action as a response. The examples shown at the link above can take a shell command (s3 sync, for example) that can be executed when a change in the directory is detected.

The inotify_simple project is another alternative. There are many options.

Upvotes: 0

Related Questions