CODEWITHSUNDEEP
pythonpython-3.x
alesplin
alesplin

Reputation: 1322

Python 3 subprocess.PIPE output error

I have a simple script that I use to automate CLI calls to our software (the Moab Workload Manager) in testing, to avoid having to use the '--xml' flag to get xml output and then pipe it through tidy so it's easily readable. It uses a subprocess.Popen call to run the command, then uses str.strip() and str.replace() to do a minor cleanup on the returned xml to make it easy to visually inspect. The code in question is here:

cmdString = "%s --xml" % cmd
cmdList = cmdString.split()

cmdRun = subprocess.Popen(cmdList,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE)

crOut,crErr = cmdRun.communicate()

xmlOutput = crOut.strip().replace("><",">\n<").replace("\" ","\"\n")

When I run this (I recently upgraded my Python to Python 3.1.2) I now get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/xmlcmd", line 50, in <module>
    runXMLCmd(getCmd())
  File "/usr/local/bin/xmlcmd", line 45, in runXMLCmd
    xmlOutput = crOut.strip().replace("><",">\n<")
TypeError: expected an object with the buffer interface

It appears that the communicate() call is returning byte arrays, but in the python interpreter, if I do dir(bytes) I can still see the strip() and replace() functions. Anybody know how to make this right?

Thanks.

Upvotes: 2

Views: 2643

Answers (3)

Jonathan Cline IEEE
Jonathan Cline IEEE

Reputation: 705

In python 3.2, using decode('ascii') fixed some unicode and type errors which were difficult to trace. Regardless of byte or bytearray the decode will convert to the string as desired.

pipe = subprocess.Popen("cmd", 1024, stdout=subprocess.PIPE)
while pipe.returncode == None:
    lines = pipe.communicate()[0]
    lines = lines.decode('ascii')
    for line in lines.splitlines():
        if (re.search('^---', line)):
            pass # do stuff

From the manual,

bytes.decode(encoding="utf-8", errors="strict") 
bytearray.decode(encoding="utf-8", errors="strict") 
Return a string decoded from the given bytes. 
Default encoding is 'utf-8'. errors may be given to set a 
different error handling scheme.

Upvotes: 1

jfs
jfs

Reputation: 414129

bytes.replace() expects bytes as arguments:

crOut.strip().replace(b"><", b">\n<").replace(b"\" ", b"\"\n")

Though in general it could be preferable to decode the input to unicode text as early as possible. And the transformations to be performed on the text (not bytes).

Upvotes: 3

sarshad
sarshad

Reputation: 331

You need to use crOut.decode("utf-8") and do the .replace in the returned string.

Upvotes: 1

Related Questions