CODEWITHSUNDEEP
pythonpopensubprocess
Mikael Grönfelt
Mikael Grönfelt

Reputation: 155

Create an empty Popen() object in Python?

Is there any way to create an empty Popen() object, or perhaps some other way to solve what I'm trying to do.

I'll give a code example to show what I'm trying to accomplish:

com = [["ls","/"],["wc"]]
p = Popen(["echo"])
for c in com:
     p = Popen(c, stdin=p.stdout,stdout=PIPE,stderr=PIPE)
l = list(p.communicate())

I have a nested list containing system commands that I'd like to run, by iterating through the list, but for this to work p must of course exist at the start of the first iteration. I can solve this like I've done, by simpy issuing an echo, but I was wondering if there's a neater and more correct way?

Upvotes: 1

Views: 2769

Answers (3)

alexis
alexis

Reputation: 50190

If all you want is to run a pipeline (= one or more |-separated commands) that is passed to your function at runtime, no need to go to all that trouble: Just use subprocess.Pipe(pipeline, shell=True). It will handle the pipes by itself. ("pipeline" is a single string, e.g. "ls" or "ls / | wc")

If for some reason you really want to start these as separate pipes, your question boils down to this: You have a loop but you need an initial value to kick it off. Instead of wasting a call to Popen, I'd do it like this:

com = [["ls","/"],["wc"]]
p = None
for c in com:
    p = Popen(c, stdin=(p.stdout if p else None), stdout=PIPE, stderr=PIPE)

(PS. I ditched my old answer since I had misunderstood what you are doing).

Upvotes: 1

torek
torek

Reputation: 488183

What you want here is (apparently) a series of pipes:

com[0] | com[1] | ... | com[n-1]

The simplest method, if you don't have to worry about "bad" characters and shells, is to just join them all up into one big shell command:

p = subprocess.Popen(' | '.join(' '.join(words) for words in com), shell=True, ...)

Alternatively, since you want stdout=None initially, you can use @pst's trick of simply having p initially be any object with a .stdout of None. But you should also note that you are depending on the system to close out each of your previous subprocess.Popen()s (so that their output pipes are close()d) inside the loop, which happens with CPython but not with Jython (as I understand it—I have not actually used Jython). So you might want a more explicit loop:

# assumes len(com) >= 1
p0 = subprocess.Popen(com[0], stdout=subprocess.PIPE)
# don't use stderr=subprocess.PIPE -- if you want stderr to be piped too
# use stderr=subprocess.STDOUT, or create a stderr pipe early manually and
# supply the fd directly; see hints in text below.
for words in com[1:]:
    p1 = subprocess.Popen(words, stdin=p0.stdout, stdout=subprocess.PIPE)
    # again you don't want to set stderr to subprocess.PIPE here
    p0.stdout.close() # now that it's going to p1, we must ditch ours
    p0 = p1 # p1 becomes the input to any new p1 we create next time
# at this point, use p0.communicate() to read output

If you redirect stderr=subprocess.STDOUT in each Popen(), this will pipe the error output from each sub-command to the next one. To pipe them all to your program, you will have to first create an OS-level pipe object, then connect each to that (single) pipe's write end, then sneak around the subprocess module to get the "final" subprocess object pointing to it, so that its select/poll code will suck any data out of that pipe. (Since subprocess.communicate() will only read-multiple from two such pipes, you cannot capture each individual sub-command's stderr output independently.)

Note: none of the above is tested...

Upvotes: 1

John Zwinck
John Zwinck

Reputation: 249153

On Unix it might be slightly preferable to use true rather than echo. true is a program which does nothing and always succeeds (see the man page for a chuckle).

Upvotes: 1

Related Questions