CODEWITHSUNDEEP
pythonencodingsubprocessstdoutpopen
F1sher
F1sher

Reputation: 7310

Reading from TextIOWrapper causes UnicodeDecodeError

I try to read subprocess line by line:

proc = subprocess.Popen(self.monitor_logcat_cmd, shell=True, stdout=subprocess.PIPE,
                        bufsize=1, universal_newlines=True)

while proc.poll() is None:
    line = proc.stdout.readline()
    print("Process line: " + str(line))

It works, yet at some point I get error:

Exception in thread Thread-14:
Traceback (most recent call last):
  File "/Users/F1sherKK/anaconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/Users/F1sherKK/Dev/Python/AutomationTestSupervisor/session/SessionThreads.py", line 46, in run
    line = proc.stdout.readline()
  File "/Users/F1sherKK/anaconda3/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 89: invalid start byte

Is there any way to add/specify encoding for stdout of subprocess? I would like to add error "ignoring".

Is there any other way to fix this?

Upvotes: 1

Views: 2423

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1123042

You could just set the errors keyword argument to Popen() to 'ignore'. From the documentation:

If encoding or errors are specified, or universal_newlines is true, the file objects stdin, stdout and stderr will be opened in text mode using the encoding and errors specified in the call or the defaults for io.TextIOWrapper.

However, it is clear your process doesn't use UTF-8 to encode its output. You may want to figure out if a) it can be configured to produce a different encoding, or b) what encoding is used and configure that instead (using the encoding keyword argument to Popen()).

Upvotes: 2

Related Questions