CODEWITHSUNDEEP
pythonlinuxshellsftp
ronald mcdolittle
ronald mcdolittle

Reputation: 567

Getting a list of files recursively using SFTP in shell or Python with no additional libraries

I am transferring files between two servers via SFTP using python's subprocess module. The only way I can connect to the remote server is via an SFTP connection.

I need to verify that the two directories on the local and remote server are identical after the transfer. This is pretty easy on the local server, a basic find command gives me what I need. However I have no clue how to get a similar result on the remote server.

Here's an example of the file structure, its identical on both machines.

JobDirectory
    Job1
        test.txt
        tonks.txt
    Job2
        wildlife.txt
    Job3
        jackinthebox.txt
        happygilmore.txt
        sadprogrammer.txt

So I need a command that'll get the filenames from Job1, Job2, and Job3 and return them to me.

Something like

echo "ls *.txt" | sftp -q [email protected]:/path

doesn't track too well here, since it needs a specific path. I could get a list of folders within the directory and run the sftp command against each of them, but that's a lot of remote connections.

The only remote access tools I can use are subprocess and Python's OS module. Something like Paramiko SFTP is not available.

Upvotes: 1

Views: 2908

Answers (1)

Martin Prikryl
Martin Prikryl

Reputation: 202282

For an easy but inefficient solution, see the answer by @pasabaporaqui to List recursively all files on sftp.

With your obscure limitation, the only solution that uses one connection would be:

  • open an sftp subprocess in Python
  • feed sequence of ls commands to it, one for each directory
  • parsing the directory listings coming on standard output, producing more ls commands for each subdirectory found.

Upvotes: 1

Related Questions