CODEWITHSUNDEEP
dockersshdeploymentdevopsrsync
xakepp35
xakepp35

Reputation: 3232

How to "rsync" docker image to remote host over ssh?

In a deployment bash script, I have two hosts:

And I need to transfer locally built docker image to $REMOTE_HOST, in most efficient way (fast, reliable, private, storage-friendly). Up to day, I have following command in my streaming script:

docker save $IMAGE_NAME :latest | ssh -i $KEY_FILE -C $REMOTE_HOST docker load

This has following PROS:

However, the CONS are also on checkerboard: When you are involved in transferring larger images, you dont know operation progress. So you have to wait unknown, but sensible time, that you cant estimate. I heard that progress can be tracked with kinda rsync --progress command

But rsync seems to transfer files, and is not working good with my ol'UNIX-style commands. Of couse you can docker load from some file, but how to avoid it?

How can I utilize piping, to preserve above advantages? (Or is there another special tool do copy build image to remote docker host, which shows progress?)

Upvotes: 3

Views: 3026

Answers (1)

Kenster
Kenster

Reputation: 25439

You could invoke pv as part of your pipeline:

docker save $1:latest | pv [options...] | ssh -i $3 -C $2 docker load

pv works like cat, in that it reads from its standard input and writes to its standard output. Except that, like the documentation says,

pv allows a user to see the progress of data through a pipeline, by giving information such as time elapsed, percentage completed (with progress bar), current throughput rate, total data transferred, and ETA.

pv has a number of options to control what kind of progress information it prints. You should read the documentation and choose the output that you want. In order to display a percentage complete or an ETA, you will probably need to supply an expected size for the data transfer using the -s option.

Upvotes: 4

Related Questions