CODEWITHSUNDEEP
pythonlistsubprocess
Souvik Ray
Souvik Ray

Reputation: 3028

How to capture output of a command run in python3 in python2?

I have a file which connects a database and fetches the result. Now the file must be ran using python 3 and my project uses python 2.7. So I run the file as a command line using subprocess module. Here is how I call the file.

import subprocess
import ast

def execute_python3(param):
    param = param.replace("\\", "")
    param = "\"" + param + "\""

    cmd = "python3 " + "get_db_result.py" + " " + param

    result = subprocess.check_output(cmd, shell=True)
    return ast.literal_eval(result)

execute_python3(sql_query)

Here in the command, I am passing sql query to the get_db_result file.

The get_db_result.py file looks something like this

import sys
def get_result():
    param = sys.argv[1]
    '''
    Logic to get result from db
    '''
    result = db_output
    print(result)

if __name__ == "__main__":
    get_result()

Now the issue is when I fetch the output from db, I have to do a print for the output to be captured by the subprocess module. This makes it difficult to parse the output to be used by program for further work. For example, when I receive an output like this

"[(u'Delhi', 20199330), (u'Mumbai', 134869470), (u'Kolkata', 6678446)]"

This is a string list of tuples which can be converted to list of tuples by doing something like ast.literal_eval(result)

But sometimes I get output like this

"[(datetime.date(2019, 5, 27), 228.168093587), (datetime.date(2019, 5, 28), 228.834493641)]"

Here ast doesn't understand datetime. Even json.loads() doesn't work on this.

How can I capture the output from a file without having to use print and simply return it back to subprocess as it is. Is it even possible?

Upvotes: 1

Views: 172

Answers (1)

BlackJack
BlackJack

Reputation: 4689

You need to serialize and deserialize the data on both ends. Simplest solution would be to use Python's pickle module and hope the types that are serialized on the Python 3 end, are similar enough to those on the deserializing Python 2 end. You need to specify the used protocol on the sending end to a version understood by the receiving end:

Receiver with safer call of subprocess (no shell process in between):

#!/usr/bin/env python
import pickle
import subprocess


def execute_python3(param):
    result = subprocess.check_output(['python3', 'get_db_result.py', param])
    return pickle.loads(result)


def main():
    execute_python3(sql_query)


if __name__ == '__main__':
    main()

Sender, explicitly choosing a pickle protocol still understood by Python 2:

#!/usr/bin/env python3
import sys
import pickle


def get_result():
    param = sys.argv[1]
    '''
    Logic to get result from db
    '''
    result = db_output
    pickle.dump(result, sys.stdout.buffer, protocol=2)


if __name__ == '__main__':
    get_result()

If this doesn't work because of differences in the (de)serialized objects between Python 2 and 3, you have to fall back to explicitly (de)serialize the data, for example in JSON, as suggested by a comment from Jay.

Upvotes: 1

Related Questions