Rsync auto local folder with GitHub folder

Question

I know that you can run Rsync commands to send folders and files to another folder and file, but I am trying to work out the best way to sync local files to my GitHub local folder due to where I develop and where my GitHub app has filed my folders are in two different locations.

1. Developer folder for example is /user/(username)/nodejs/(app name)

2. GitHub folder is /user/(username)/github/(git name)

Is there anyway to have the two folders synced?

For example, if I make a change in my (app name) folder I want it to update in the GitHub folder.

If a pull request from GitHub which updates the (git name) folder I want that sent to the (app name) folder.

I know that in basic you could do:

rsync -a source dest

However that will copy everything.

Also the other issue is what is the pull request has a bug in it (I know as a developer I can fix the bug and then push that fix to the GitHub but I don't want to run into a situation where I potentially screw up my (app name) folder to the point I have to get the latest version from GitHub and re code work I have done.

I also know I could use something like this:

rsync -v -a --ignore-existing /(APP NAME)/ /(GIT NAME)/

Answer 1

Just wanted to throw this into the mix in case it helps... I had to do something very similar not so long ago. I didn't want to add the directory in question to the git repo because it was 4GB, however in my situation the two servers were remote.

Rather than using rsync... which is a great approach, I went for AWS S3... it was really easy to sync to, and just needed a bit of tinkering with IAM to get the permissions exactly dialled in.

... then using the AWS CLI i used the S3 sync feature which easily cleans up any deleted files and the like.

To upload your files:

aws s3 sync /path/to/source_folder/ s3://my-s3-bucket/git-project/whatever/

Then to download on dev servers or production:

aws s3 sync s3://my-s3-bucket/git-project/whatever /path/to/source_folder/

... I was then able to link the S3 bucket to my CDN (CloudFront) to serve these larger files.

I hope this helps.

Answer 2

please try this, a good remote sync with delete any additional removed source files, a good mirror for compression add -z flag, --delete-delay wait till files are copied...

Simple quick answer, ignores existing files, fast, cleans up old files, waits for completion, recursive etc.

rsync -rtvu --delete-delay source_folder/ destination_folder/

---

Dry Runs/Testing:

I realize you didn't ask, but to help your case, may I recommend a dry run [A-archive,n-DryRun,V-Verobse], -a makes the copy recursive and keeps original props like modification times, while also copying the symlinks that it encounters as symlinks, preserve the permissions, preserve the owner and group information, and preserve device and special files...

//[A-archive,n-DryRun,V-Verobse]
rsync -anv someDir1/ someDestdir2

// now compare those two
rsync -anv someDir1 someDestdir2   

---

When dealing with Remote Servers

1 With SSH

*In some cases, if you are `rsync`ing between two servers, wordpress hosting etc, rsync doesn't manage/have the connection between the two servers, you can ssh into them and run this*

// rsync between two remote servers, if you have ssh, this i
ssh -R 50000:host2:22 host1 'rsync -e "ssh -p 50000" -vuar /host1/someDir1 localhost:/host2/someDestdir2'

Additional Misc. for Remote servers with IP

2 Local folder to remote folder, using a domain, an IP address and a server defined in the SSH configuration file:

rsync -rtvz source_folder/ user@domain:/path/to/destination_folder/
rsync -rtvz source_folder/ user@xxx.xxx.xxx.xxx:/path/to/destination_folder/
rsync -rtvz source_folder/ server_name:/path/to/destination_folder/

3 Remote folder to local folder, using a domain, an IP address and a server defined in the SSH configuration file:

rsync -rtvz user@domain:/path/to/source_folder/ destination_folder/
rsync -rtvz user@xxx.xxx.xxx.xxx:/path/to/source_folder/ destination_folder/
rsync -rtvz server_name:/path/to/source_folder/ destination_folder/