CODEWITHSUNDEEP
pythonsshsftpparamiko
user1452759
user1452759

Reputation: 9450

Transfer multiple files in a directory on a local server to different directories on remote server in Python

There are some files inside a directory on an Informatica Server. I need to transfer these files onto another server based on their file names into different directories.

Say there are 2 files, a.dat_1 and b.dat_2 inside a directory called low on the Informatica server.

I need to transfer this low directory onto another server where the file a.dat_1 goes to a directory say, local and b.dat_2 goes to another directory called local2. This needs to be done in Python.

I have used Paramiko to do simple transfers but not directories. And not in separate directories like local and local2.

Upvotes: 1

Views: 2651

Answers (3)

tnknepp
tnknepp

Reputation: 6253

I do something similar for transferring files daily. As yakxxx suggests, I zip my files then transfer. Example of what I do (on Windows machines):

import zipfile
from glob import glob as gg

files = gg('path*.txt')
# open zip file (create it, or open if already exists)
zFile = zipfile.ZipFile('FileName.zip','w') 

# zip files on local machine
[zFile.write(r,r,zipfile.ZIP_DEFLATED) for r in files]

Alternately, if the zipfile already exists and you want to add new files.

zFile = zipfile.ZipFile('FileName.zip','a')

# List files already zipped.  
done = zipfile.ZipFile('FileName.zip','r').namelist()

# zip file into zip file.  
[zFile.write(r,r,zipfile.ZIP_DEFLATED) for r in files if r not in done]

Now push this zip file to your remote machine via paramiko.

---------------

EDIT

FYI, when writing to to zip files you need to be careful. I wrote this assuming you are running within the directory that has the files you want to zip. If you are not, you need to use:

import os
zFile.write(r,os.path.basename(r),zipfile.ZIP_DEFLATED)

Upvotes: 1

marklap
marklap

Reputation: 481

Given this SO Directory transfers on paramiko; you won't be able to copy a directory as a bulk operation.

If it were me, I'd create a temp folder and start by copying the source files into a directory structure compatible with your destination requirements. Then compress them as yakxxx suggests, send the compressed file over the wire with Paramiko SFTP and uncompress on the other end with Paramiko SSH.

Upvotes: 1

yakxxx
yakxxx

Reputation: 2941

You can make a tar archive out of your directory. For this use shell command 

tar -cvf archive_name.tar directory_name

then transfer this archive to another machine and untar it there:

tar -xvf archive_name.tar

You could also uze gzip or other compressor on your tarred archive to make the transfer faster.

Upvotes: 1

Related Questions