CODEWITHSUNDEEP
pythonshell
hyperion007
hyperion007

Reputation: 75

execute "shell" command from python with subprocess & multiprocessing

I am currently launching the serials shell commands from python with subprocess and multiprocessing. here code is not real one, but similar :

    def fakeFunc(cmd):
        print("the pid "+str(os.getpid())+"begin", end=":")
        process=Popen(cmd, stdout=PIPE, shell=True, stderr=STDOUT)
        for line in iter(process.stdout.readline,b''):
            line_str=line.decode(sys.stdout.encoding)
            ##### analyse line_str######
    with Pool() as pool:
        cmds=['mkdir -p /home/jeff/workspace/zebu/compile/vdv && cd /home/jeff/workspace/zebu/compile/vdv && compile', 'mkdir -p /home/jeff/workspace/zebu/compile/vdr && cd /home/jeff/workspace/zebu/compile/vdr && compile', 'mkdir -p /home/jeff/workspace/zebu/compile/rdv && cd /home/jeff/workspace/zebu/compile/rdv && compile']
        pool.map_async(fakeFunc, cmds)
        pool.close()
        pool.join()

it seems that only the last one in cmds has been executed, other three ones jump out of function after the print("the pid "+str(os.getpid())+"begin", end=":"). any help is appreciated

Upvotes: 0

Views: 110

Answers (1)

Yaroslav Surzhikov
Yaroslav Surzhikov

Reputation: 1608

Wrap body of your fakeFunc into try-except with cathing Exception, so you could find what's wrong is going on there.

def fakeFunc(cmd):
    try:
        print("the pid " + str(os.getpid()) + " begin", end=":\n")
        process = Popen(cmd, stdout=PIPE, shell=True, stderr=STDOUT)
        for line in iter(process.stdout.readline, b''):
            line_str = line.decode(sys.stdout.encoding)
            ##### analyse line_str######
    except Exception as e:
        print(e)

with Pool() as pool:
    cmds=['mkdir -p /home/jeff/workspace/zebu/compile/vdv && cd /home/jeff/workspace/zebu/compile/vdv && compile', 'mkdir -p /home/jeff/workspace/zebu/compile/vdr && cd /home/jeff/workspace/zebu/compile/vdr && compile', 'mkdir -p /home/jeff/workspace/zebu/compile/rdv && cd /home/jeff/workspace/zebu/compile/rdv && compile']
    pool.map_async(fakeFunc, cmds)
    pool.close()
    pool.join()

Upvotes: 1

Related Questions