Unexpected behaviour when calling shell command on a deployed web app

Question

I am deploying a flask app, in which a multiprocessing.Process will be started. Inside this process I call a shell command trough subprocess.call(). The script runs fine when executed on local host, when deployed with nginx and gunicorn, the flask app behaves as expected until the subprocess starts, then I recieve the following error log:

DEBUG:root:start to run command
DEBUG:root:(<class 'FileNotFoundError'>, FileNotFoundError(2, "No such file or directory: 'java -jar ábsolute/path/to/jar/file')

                                Process(
                                target=decode_upload,
                                args=(
                                        path_to_blf,
                                        path_to_dbc_folder,
                                        path_to_splitted,
                                        path_to_decoder,
                                        system_layout,
                                        dc_doc,
                                        dc_id,
                                        file_type,
                                )
                            ).start()

Here is the rellevant part of the function.

def decode_file(
    path_to_blf,
    path_to_dbc_folder,
    path_to_splitted,
    path_to_decoder,
    system_layout=DEFAULT_SYSTEM_LAYOUT):
    command = "{} {} --blf={}".format(
                                        SOFTWARE_COMMAND,
                                        path_to_decoder,
                                        path_to_blf
    )
    for dbc_file_name in DBC_FILE_NAME_LIST:
        command += " --dbc={}".format(
                                    os.path.join(
                                            path_to_dbc_folder,
                                             dbc_file_name
                                    )
        )
    command += " --out={}".format(path_to_splitted)


    logging.debug("start to run command")
    subprocess.call(command)
    logging.debug(f)
    logging.debug("run command end")

def decode_upload(
                        path_to_blf,
                        path_to_dbc_folder,
                        path_to_splitted,
                        path_to_decoder,
                        system_layout,
                        dc_doc,
                        dc_id,
                        file_type):
    logging.basicConfig(filename='flask.log',level=logging.DEBUG)
    logging.debug('This message should go to the log file')

    try:
        decode_file(
            path_to_blf,
            path_to_dbc_folder,
            path_to_splitted,
            path_to_decoder,
            system_layout)
    except:
        logging.debug(sys.exc_info())

The process fails when it gets to this line.

subprocess.call(command)

if I try to call the "command" from the command line it works without a problem.

Answer 1

The proper solution is to pass the command as a list of parsed arguments. You need to understand how the shell handles quoting and argument splitting to do this right.

As a quick cheat,

printf "'%s'\n" your command line here

in your shell should give you a good idea of how the shell expands the arguments. For example

bash$ printf "'%s'\n" java -jar "/path/complex path with spaces.jar" \* \>\<
'java'
'-jar'
'/path/complex path with spaces.jar'
'*'
'><'

shows you that you need

subprocess.call(['java', '-jar', '/path/complex path with spaces.jar', '*', '><'])

Adapting your code for this, we obtain

command = [SOFTWARE_COMMAND, path_to_decoder, '--blf={}'.format(path_to_blf)]
for dbc_file_name in DBC_FILE_NAME_LIST:
    command.append("--dbc={}".format(
        os.path.join(path_to_dbc_folder, dbc_file_name)))
command.append("--out={}".format(path_to_splitted))

Answer 2

from subprocess import Popen
command='your complete command as you paste on cmd'
p1=Popen(command,Shell=True)

This will help you to run the command as complete string