CODEWITHSUNDEEP
pythonsshsftpparamiko
Hick
Hick

Reputation: 36394

How to transfer data from one server to another without downloading to local using Python?

I'm using Paramiko to download a few images to localhost and then upload them to a server via SSH.

How to achieve it without downloading to local? As the server times out while downloading and uploading large files (> 100 mb).

Something similar to this: https://unix.stackexchange.com/questions/9029/how-to-wget-a-file-to-a-remote-machine-over-ssh

But in Python.

Upvotes: 0

Views: 2204

Answers (1)

Austin Phillips
Austin Phillips

Reputation: 15746

Assuming you have a network structure like this:

local machine ---X--- machine A
              |            |
              |            Z
              |            |
              ---Y---- machine B

then you're currently downloading over link X, then uploading over link Y. If machine A can talk directly to machine B, then link Z also exists. This would be the case where both machine A and machine B are publicly accessible.

So you want to have machine A initiate a direct transfer of images to machine B. At this point, it's the code running on machine A/B which is handling the transfer, so it's likely that a Python solution would only help you get the transfer started.

If you were doing this from the bash shell using ssh, you might type something like this:

ssh user@machineA 'scp myfile user@machineB'

This assumes that you have shell access to the ssh server on machine A and that it has the scp program installed. Paramiko is only useful for making the connection to machine A to initiate the transfer, not for handling the transfer itself.

Something like this might work:

ssh = paramiko.SSHClient()
ssh.connect('machineA', username='user', password='passwd')
stdin, stdout, stderr = ssh.exec_command("scp '{imgpath}' user@machineB".format(
    imgpath='path/to/file/on/machineA')

Update:

If the images are being served from machine A but you don't have ssh login access, then the images can be pulled directly to machine B by logging into machine B and executing a command to fetch the image. wget can typically be used for this. So the code to do this from localhost might be:

ssh = paramiko.SSHClient()
ssh.connect('machineB', username='user', password='passwd')
stdin, stdout, stderr = ssh.exec_command("wget '{imgurl}'".format(
    imgurl='http://url.to.image/file')

For testing, just skip the use of paramiko and use ssh/wget directly.

Upvotes: 1

Related Questions