CODEWITHSUNDEEP
pythonlinuxpython-2.7popenchaining
user1260503
user1260503

Reputation:

Using && in subprocess.Popen for command chaining?

I'm using subprocess.Popen with Python, and I haven't come across an elegant solution for joining commands (i.e. foobar&& bizbang) via Popen.

I could do this:

p1 = subprocess.Popen(["mmls", "WinXP.E01"], stdout=subprocess.PIPE)
result = p1.communicate()[0].split("\n")
for line in result:
    script_log.write(line)

script_log.write("\n")

p1 = subprocess.Popen(["stat", "WinXP.E01"], stdout=subprocess.PIPE)
result = p1.communicate()[0].split("\n")
for line in result:
    script_log.write(line)

But that really isn't very aesthetically pleasing (especially if I'm daisy-chaining multiple commands via Popen.

I'd like to replicate this output in as few command blocks as possible.

not@work ~/ESI/lab3/images $ mmls WinXP.E01 && echo -e "\n" && stat WinXP.E01
DOS Partition Table
Offset Sector: 0
Units are in 512-byte sectors

     Slot    Start        End          Length       Description
00:  Meta    0000000000   0000000000   0000000001   Primary Table (#0)
01:  -----   0000000000   0000000062   0000000063   Unallocated
02:  00:00   0000000063   0020948759   0020948697   NTFS (0x07)
03:  -----   0020948760   0020971519   0000022760   Unallocated


  File: `WinXP.E01'
  Size: 4665518381  Blocks: 9112368    IO Block: 4096   regular file
Device: 14h/20d Inode: 4195953     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    nott)   Gid: ( 1000/    nott)
Access: 2013-03-16 23:20:41.901326579 -0400
Modify: 2013-03-04 10:05:50.000000000 -0500
Change: 2013-03-13 00:25:33.254684050 -0400
 Birth: -

Any suggestions?

Note: I'd like to avoid typing this into subprocess.Popen

p1 = subprocess.Popen(["mmls WinXP.E01 && echo -e '\n' && stat WinXP.E01"], stdout=subprocess.PIPE)

Upvotes: 11

Views: 16499

Answers (3)

arthur.sw
arthur.sw

Reputation: 11619

Using shell=True is strongly discouraged when the command is constructed from external input ; so another way to chain multiple commands is to create a dedicated batch file:

import tempfile
import platform

commands = ['conda activate envName', 'python script.py']

is_windows = platform.system() == 'Windows'

# Create a temporary batch file containing one command per line and execute it
with tempfile.NamedTemporaryFile(suffix='.bat' if is_windows else '.sh', mode='w') as tmp:
    tmp.write('\n'.join(commands))
    tmp.flush()
    execute_file = [tmp.name] if is_windows else ['/bin/bash', tmp.name]
    # Give execution permissions on unix platforms 
    if not is_windows:
        subprocess.run(['chmod', 'u+x', tmp.name])
    subprocess.run(execute_file)

Note: I did test on Windows yet

Upvotes: 0

Matt
Matt

Reputation: 442

&& is a shell operator, Popen does not use a shell by default.

If you want to use shell functionality use shell=True in your Popen call, but be aware that it is slightly slower/more memory intensive.

p1 = subprocess.Popen(["mmls", "WinXP.E01", "&&", "echo", "-e", "\"\n\"", "&&", "stat", "WinXP.E01"],
                      stdout=subprocess.PIPE, shell=True)

Upvotes: 10

Blckknght
Blckknght

Reputation: 104682

How about this:

from subprocess import Popen, PIPE

def log_command_outputs(commands):
    processes = [Popen(cmd, stdout=PIPE) for cmd in commands]
    outputs = [proc.communicate()[0].split() for proc in processes]
    for output in outputs:
        for line in output:
            script_log.write(line)
        script_long.write("\n")

This starts the commands in parallel, which might make it a little faster than doing them one by one (but probably not by a large margin). Since the communicate calls are sequential though, any command that has a large output (more than a pipe's buffer) will block until it's turn comes to be cleaned up.

For your example command chain, you'd call:

log_command_outputs([["mmls", "WinXP.E01"], ["stat", "WinXP.E01"]])

Upvotes: 2

Related Questions