CODEWITHSUNDEEP
pythonsshscp
user1452759
user1452759

Reputation: 9460

Python script to get files from one server into another and store them in separate directories?

I am working on server 1. I need to write a Python script where I need to connect to a server 2 and get certain files (files whose name begins with the letters 'HM') from a directory and put them into another directory, which needs to be created at the run time (because for each run of the program, a new directory has to be created and the files must be dumped in there), on server 1.

I need to do this in Python and I'm relatively new to this language. I have no idea where to start with the code. Is there a solution that doesn't involve 'tarring' the files? I have looked through Paramiko but that just transfers one file at a time to my knowledge. I have even looked at glob but I cannot figure out how to use it.

Upvotes: 6

Views: 29013

Answers (4)

jfs
jfs

Reputation: 414685

You could use fabric. Create fabfile.py on server1:

import os
from fabric.api import get, hosts

@hosts('server2')
def download(localdir):
    os.makedirs(localdir)  # create dir or raise an error if it already exists 
    return get('/remote/dir/HM*', localdir)  # download HM files to localdir

And run: fab download:/to/dir from the same directory in a shell (fabfile.py is to fab as Makefile is to make).

Upvotes: 0

pythonian29033
pythonian29033

Reputation: 5207

to transfer the files you might wanna check out paramiko

import os
import paramiko

localpath = '~/pathNameForToday/'
os.system('mkdir ' + localpath)
ssh = paramiko.SSHClient() 
ssh.load_host_keys(os.path.expanduser(os.path.join("~", ".ssh", "known_hosts")))
ssh.connect(server, username=username, password=password)
sftp = ssh.open_sftp()
sftp.get(remotepath, localpath)
sftp.close()
ssh.close()

I you wanna use glob you can do this:

import os
import re
import glob

filesiwant = re.compile('^HM.+') #if your files follow a more specific pattern and you don't know regular expressions you can give me a sample name and i'll give you the regex4it
path = '/server2/filedir/'
for infile in glob.glob( os.path.join(path, '*') ):
    if filesiwant.match(infile):
         print "current file is: " + infile

otherwise an easier alternative is to use os.listdir()

import os
for infile in os.listdir('/server2/filedir/'):
    ...`

does that answer your question? if not leave comments

Upvotes: 4

John La Rooy
John La Rooy

Reputation: 304393

Just use ssh and tar. No need to get Python involved

$ ssh server2 tar cf - HM* | tar xf -

The remote tar can pipe straight into the local tar

Upvotes: 0

Joe
Joe

Reputation: 2438

Python wouldn't be my first choice for this task, but you can use calls to the system and run mkdir and rsync. In particular you could do

import os
os.system("mkdir DIRECTORY")
os.system("rsync -cav user@server2:/path/to/files/HM* DIRECTORY/")

Upvotes: 1

Related Questions