CODEWITHSUNDEEP
pythonbashscp
evanjd
evanjd

Reputation: 415

Passing a filename with an apostrophe into scp using python

I'm trying to write a python script to copy files from a remote server to a local directory via scp.

Because I'm running this on an OpenELEC distribution (minimal HTPC linux distro, read-only filesystem except for userhome makes it impractical to install python ssh module), I'm doing this ugly and just passing the filename to the scp command via os.system.

SCPCopy = "scp -c blowfish -C user@host:\"" + pipes.quote(file) + "\" /storage/downloads/incoming/"
SCPCopy = SCPCopy.replace('\n','')
os.system(SCPCopy)

This works, except for filenames containing an apostrophe.

Below is an example of what gets passed to os.system in a file with an apostrophe:

scp -c blowfish -C user@host:"'/media/sdi1/home/data/bob'"'"'s file.avi'" /storage/downloads/incoming/

And the error:

sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file

It looks pipes.quote(x) is escaping the apostrophe (as it should), but obviously the syntax is still incorrect. I've experimented ditching pipes.quote(x) and replacing apostrophes with /' but that isn't getting me anywhere either.

Upvotes: 2

Views: 4458

Answers (1)

Jo So
Jo So

Reputation: 26531

As scp is based on SSH, the filenames you give to it are subject to shell escaping on the remote side as well. Thus you need to escape twice.

A correctly escaped cmdline for the shell:

scp -c blowfish -C user@host:"\"/media/sdi1/home/data/bob's file\"" /storage/.../

To make a python string, we have to add one more level of escaping. To stay sane, we could use triple-quotes:

"""scp -c blowfish -C user@host:"\"/media/sdi1/home/data/bob's file\"" /storage/.../"""

If you do it programmatically (e.g. using the deprecated pipes.quote), then don't touch the filename at all (in your example above, you added apostrophes around the filename).

fp = "/media/sdi1/home/data/bob's file.avi"
fp = "user@host:" + pipes.quote(pipes.quote(fp))

cmdline = "scp -c blowfish -C " + fp + " /storage/downloads/incoming/"
os.system(cmdline)

This is admittedly confusing. For a simple model, the whole point of pipes.quote is to escape the input so that the input will be parsed by the shell as exactly one word, which is equal to the input.

The following is a more generally correct way (and yields the same result):

fp = "/media/sdi1/home/data/bob's file.avi"
# the filepath argument escaped for ssh/scp on the remote side
fp = pipes.quote(fp)
commandargs = ["scp", "-c", "blowfish", "-C", "user@host:"+fp, "/storage/downloads/incoming/"]
# escape all words for the local shell, and then concatenate space-separated
cmdline = " ".join(map(pipes.quote, commandargs))
os.system(cmdline)

It expresses more clearly the intent: Controlling what words exactly the shell will parse.

But why start with a shell in the first place? We don't need one and can save the escaping on the local side. To spawn a process with our args, directly, use commands from the os.exec* family.

fp = pipes.quote("/media/sdi1/home/data/bob's file.avi")
commandargs = ["scp", "-c", "blowfish", "-C", "user@host:"+fp, "/storage/downloads/incoming/"]
if os.fork() == 0:
    os.execvp("scp", commandargs)

Upvotes: 6

Related Questions