CODEWITHSUNDEEP
google-cloud-storagegsutil
fredrik
fredrik

Reputation: 10281

How to perform logging with gsutil rsync

What's the proper way to log any errors or warnings when performing a quiet rsync?

This is what I currently run from my crontab:

gsutil -m -q rsync -r -C /mount1/share/folder gs://my-bucket-1/folder/ > /mount2/share/folder/gsutil.log

Since the log file is always completely empty and I'm uploading terabytes of data I'm starting to think that maybe even errors and warnings are being supressed.

Upvotes: 9

Views: 9976

Answers (1)

fredrik
fredrik

Reputation: 10281

After having realized that this is related to how you pipe stdout and/or stderr to files in general, the answer really lies within this existing thread: How to redirect both stdout and stderr to a file

So a simple solution to log as much as possible into one single log file could be something like:

gsutil -d rsync [src] [dst] &> [logfile]

...where -d enables debug output. I found this to be the only way to show files which were affected by an error such as CommandException: 3 files/objects could not be copied. Please note that -d exposes authentication credentials.

Upvotes: 10

Related Questions