CODEWITHSUNDEEP
pythonsubprocesspopen
varesa
varesa

Reputation: 2419

python: subprocess.Popen() behaviour

I am trying to use rsync with python. I have read that the preferred way to passing arguments to Popen is using an array.

The code I tried:

p = Popen(["rsync",
        "\"{source}\"".format(source=latestPath),
        "\"{user}@{host}:{dir}\"".format(user=user, host=host, dir=dir)],
        stdout=PIPE, stderr=PIPE)

The result is rsync asking for password, even though I have set up SSH keys to do the authentication.

I think this is a problem with the environment the new process gets executed in. What I tried next is:

p = Popen(["rsync", 
   "\"{source}\"".format(source=latestPath),
   "\"{user}@{host}:{dir}\"".format(user=user, host=host, dir=dir)],
   stdout=PIPE, stderr=PIPE, shell=True)

This results in rsync printing the "correct usage", so the arguments are passed incorrectly to rsync. I am not sure if this is even supposed to work(passing an array with shell=True)

If I remove the array altogether like this:

p = Popen("rsync \"{source}\" \"{user}@{host}:{dir}\"".format(
        source=latestPath, user=user, host=host, dir=dir),
      stdout=PIPE, stderr=PIPE, shell=True)

The program works fine. It really doesn't matter for the sake of this script, but I'd like to know what's the difference? Why don't the other two(mainly the first one) work?

Is it just that the shell environment is required, and the second one is incorrect?

EDIT: Contents of the variables

latestPath='/home/tomcat/.jenkins/jobs/MC 4thworld/workspace/target/FourthWorld-0.1-SNAPSHOT.jar'
user='mc'
host='192.168.0.32'
dir='/mc/test/plugins/'

Upvotes: 3

Views: 1227

Answers (3)

amoffat
amoffat

Reputation: 714

Just a suggestion, it might be easier for you to use sh instead of subprocess:

import sh
sh.rsync(latestPath, user+"@"+host+":"+dir)

Upvotes: 1

Fred Foo
Fred Foo

Reputation: 363497

I'd like to know what's the difference?

When shell=True, the entire command is passed to the shell. The quotes are there so the shell can correctly pick the command apart again. In particular, passing

foo "bar baz"

to the shell causes it to parse the command as (Python syntax) ['foo', 'bar baz'] so that it can execute the foo command with the argument bar baz.

By contrast, when shell=False, Python will pass the arguments in the list to the program immediately. For example, try the following subprocess commands:

>>> import subprocess
>>> subprocess.call(["echo", '"Hello!"'])
"Hello!"
0
>>> subprocess.call('echo "Hello!"', shell=True)
Hello!
0

and note that in the first, the quotes are echoed back at you by the echo program, while in the second case, the shell has stripped them off prior to executing echo.

In your specific case, rsync gets the quotes but doesn't know how it's supposed to handle them; it's not itself a shell, after all.

Upvotes: 1

hwjp
hwjp

Reputation: 16071

Could it be to do with the cwd or env parameters? Maybe in the first syntax, it can't find the SSH keys...

Upvotes: 1

Related Questions