CODEWITHSUNDEEP
pythonpython-2.7exceptionsubprocess
Bomaz
Bomaz

Reputation: 1961

Reraise exception from subprocess

I have code executing in a subprocess that is supposed to raise an exception.

I would like to raise the same exception in the main process when the exception is returned from the subprocess (preferably while preserving the stack trace) but I am not sure how to do this.

I capture the stderr from the subprocess just fine but I can't find how to parse it so I get the type of exception. How would I accomplish this?

I am using python 2.7

main method

import subprocess

example=subprocess.Popen(["python","example.py"],
                                    stdout = subprocess.PIPE,
                                    stderr = subprocess.PIPE)
return_obj, return_error = example.communicate()

if return_error:
# replace with error from subprocess
  print "Should raise ",NameError('HiThere')
  raise TypeError('Wrong type')

subprocess

raise NameError('HiThere')

Upvotes: 7

Views: 1377

Answers (3)

Andrey
Andrey

Reputation: 6796

If both processes are python processes, and have to use pipes, exception object can be serialized through the pipe.

Caller:

parent_conn.send(event)
result = parent_conn.recv()
if isinstance(result, Exception):
    raise result
else:
    return result

Receiver:

def subprocess_loop(parent_conn, child_conn):
    
    parent_conn.close()

    while True:
        event = child_conn.recv()
        try:
            result = event_handler(event)
        except Exception as exc:
            child_conn.send(exc)
            continue
        child_conn.send(result)

Receiver process creation:

parent_conn, child_conn = Pipe()
process = Process(target=subprocess_loop, args=(parent_conn, child_conn))
process.start()
child_conn.close()

Upvotes: 0

loopbackbee
loopbackbee

Reputation: 23332

If all you want is your python code running in a separate process, you should probably not use subprocess. As Serge Ballesta said, subprocess' purpose is to run a different program at the OS level, without particularly caring for what it is - there's nothing to help you properly handle the intricacies of a python interpreter process.

For this kind of purpose, it's probably best to simply import your code and use multiprocessing, which exposes a high-level interface to help you run python code in multiple processes.

Assuming you have a well defined main function on example.py:

from examply import main as example_main
import multiprocessing
pool= multiprocessing.Pool(1)
pool.apply( example_main )

Using this code, both exceptions and return values will be transparently given to your main process.

You can also use Pool.apply_async, if you don't want to block waiting for the result.

Upvotes: 4

Serge Ballesta
Serge Ballesta

Reputation: 149155

The subprocess executes a different native application. It could be java, C++, lisp or even Fortran or Cobol. So you have only 2 ways to get the exception from a Python subprocess :

  • forget the subprocess at all, and directly call python code in the same python program, at simple try except will do the work
  • define in interface contract between the main and subprocess, for example requiring that the error output must be empty if no fatal exeption occurs and else contains a pickle of the exception and the stacktrace .

Upvotes: 1

Related Questions