CODEWITHSUNDEEP
pythonpython-3.xsubprocess
benwad
benwad

Reputation: 6594

Subprocess output after CalledProcessError in python3

I'm trying to run bandit from within Python using subprocess. I've got some code like this:

import subprocess

command = ['bandit', '-r', 'goodcode']
output = subprocess.check_output(command)
print(output.decode())

This works fine, and gives me a string output like this:

Run started:2016-09-27 10:37:17.567678

Test results:
        No issues identified.

Code scanned:
        Total lines of code: 940
        Total lines skipped (#nosec): 0

Run metrics:
        Total issues (by severity):
                Undefined: 0
                Low: 0
                Medium: 0
                High: 0
        Total issues (by confidence):
                Undefined: 0
                Low: 0
                Medium: 0
                High: 0
Files skipped (0):

...however, when I run it on a directory where bandit returns some errors, the bandit process itself returns 1. Therefore I have to catch the CalledProcessError like so:

import subprocess

command = ['bandit', '-r', 'badcode']

try:
    output = subprocess.check_output(command)
except subprocess.CalledProcessError as e:
    output = e.output

print(output.decode())

...which gives me the following result:

b"Run started:2016-09-27 10:42:26.616123\n\nTest results:\n>> Issue: [B110:try_except_pass] Try, Except, Pass detected.\n   Severity: Low   Confidence: High\n   Location: badcode/conf/settings_development.py:93\n92\t    from .settings_local import *\n93\texcept:\n94\t    pass\n\n--------------------------------------------------\n>> Issue: [B105:hardcoded_password_string] Possible hardcoded password: 'password'\n   Severity: Low   Confidence: Medium\n   Location: badcode/frontend/tests/test_views.py:21\n20\t        form['username'] = self.user.username\n21\t        form['password'] = 'password'\n22\t\n\n--------------------------------------------------\n>> Issue: [B105:hardcoded_password_string] Possible hardcoded password: 'password'\n   Severity: Low   Confidence: Medium\n   Location: badcode/frontend/tests/test_views.py:35\n34\t        form['username'] = self.user.username\n35\t        form['password'] = 'password'\n36\t\n\n--------------------------------------------------\n>> Issue: [B110:try_except_pass] Try, Except, Pass detected.\n   Severity: Low   Confidence: High\n   Location: badcode/reasons/views.py:234\n233\t                nr = subject.number\n234\t            except:\n235\t                pass\n\n--------------------------------------------------\n>> Issue: [B110:try_except_pass] Try, Except, Pass detected.\n   Severity: Low   Confidence: High\n   Location: badcode/reasons/views.py:277\n276\t                nr = event.number\n277\t            except:\n278\t                pass\n\n--------------------------------------------------\n>> Issue: [B110:try_except_pass] Try, Except, Pass detected.\n   Severity: Low   Confidence: High\n   Location: badcode/retention/migrations/0010_auto_20160527_1603.py:13\n12\t            retention.save()\n13\t        except:\n14\t            pass\n\n--------------------------------------------------\n>> Issue: [B110:try_except_pass] Try, Except, Pass detected.\n   Severity: Low   Confidence: High\n   Location: badcode/retention/migrations/0015_auto_20160623_1051.py:13\n12\t            retention.save()\n13\t        except:\n14\t            pass\n\n--------------------------------------------------\n>> Issue: [B108:hardcoded_tmp_directory] Probable insecure usage of temp file/directory.\n   Severity: Medium   Confidence: Medium\n   Location: badcode/utils/views.py:322\n321\t        css = '{}/static/badcode/css/screen.css'.format(settings.ROOT_DIR)\n322\t        location = '/tmp/{}.pdf'.format(filename)\n323\t\n\n--------------------------------------------------\n\nCode scanned:\n\tTotal lines of code: 15287\n\tTotal lines skipped (#nosec): 0\n\nRun metrics:\n\tTotal issues (by severity):\n\t\tUndefined: 0.0\n\t\tLow: 7.0\n\t\tMedium: 1.0\n\t\tHigh: 0.0\n\tTotal issues (by confidence):\n\t\tUndefined: 0.0\n\t\tLow: 0.0\n\t\tMedium: 3.0\n\t\tHigh: 5.0\nFiles skipped (0):\n"

Note that the b"" is inside the string, so output[0] == 'b' and output[1] == '"'. Why is this? Shouldn't e.output encode the output the same as output if the process had returned 0?

Upvotes: 1

Views: 723

Answers (1)

jfs
jfs

Reputation: 414795

It seems you've looked at the output variable before .decode() method is called in the second case. output refers to the same bytestring in both cases.

>>> import subprocess, sys
>>> output1 = subprocess.check_output([sys.executable, '-c', 'print("abc")'])
>>> try:
...   subprocess.check_output([sys.executable, '-c', 
...                            'print("abc"); import sys;sys.exit(1)'])
... except subprocess.CalledProcessError as e:
...   output2 = e.output
... 
>>> output1 == output2
True

If non-zero exit status is not an error in your case then you could use a function that doesn't raise CalledProcessError in such cases:

#!/usr/bin/env python3
from subprocess import run, PIPE

finished_process = run(command, stdout=PIPE, universal_newlines=True)
print("{p.returncode}, {p.stdout}".format(p=finished_process))
# e.g.: 1, abc

universal_newlines=True is an obscure spelling for "enable text mode" here.

Upvotes: 2

Related Questions