CODEWITHSUNDEEP
pythondecodeparamikoreadlines
Fragtzack
Fragtzack

Reputation: 373

paramiko-1.16.0 readlines() decode error

Doing a fairly standard paramiko implementation returns decode error on readlines() for oddly formed text from remote command. I can not change the command output. How can I write the code to properly decode. See decode error text at bottom:

code snip being used:

connect = paramiko.SSHClient()
connect.connect(self.name,
                username = self.ruser,
                password = password,
                key_filename = idkey,
                timeout = 15,
               )
stdin, stdout, stderr = connect.exec_command(cmd)
retval=stdout.channel.recv_exit_status()
stdin.flush()
stdout.flush()
stderr.flush()
out = stdout.readlines()
sys.exit()

ERROR TEXT:

File "../pylib/hosts/host.py", line 128, in cmd
out = stdout.readlines()
File "/usr/local/lib/python3.4/site-packages/paramiko/file.py", line 285, in readlines
line = self.readline()
File "/usr/local/lib/python3.4/site-packages/paramiko/file.py", line 270, in readline
return line if self._flags & self.FLAG_BINARY else u(line)
File "/usr/local/lib/python3.4/site-packages/paramiko/py3compat.py", line 148,
in u return s.decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 80: invalid continuation byte

Upvotes: 2

Views: 3108

Answers (2)

Ben Song
Ben Song

Reputation: 69

Based on Fragtzack, here is a more flexible way.

def u(s, encoding="utf8"):
    """cast bytes or unicode to unicode"""
    if isinstance(s, bytes):
        #return s.decode(encoding)
        try:
            return s.decode(encoding)
        except UnicodeDecodeError:
            return decoding(s)
    elif isinstance(s, str):
        return s
    else:
        raise TypeError("Expected unicode or bytes, got {!r}".format(s))

def decoding(s, encodings=[ "utf-8", "cp936"]):
    """cast bytes to unicode by encodings"""
    for encoding in encodings:
        try:
            return s.decode(encoding)
        except UnicodeDecodeError as lastError:
            continue
    else:
        raise lastError

Upvotes: 0

Fragtzack
Fragtzack

Reputation: 373

Well, I modified the paramiko.py3compat "u" method for python 3. (Did not modify python 2 version) If the utf8 decode fails, try ISO-8859-1. Standard try: except: wrapper.

Suspect this issue only occurs with Python 3. Would have been nice to overide the py3compay.pt u() method instead of modifying the paramiko library file, but needed to move on from this issue.

Here is the new u() method I patched into py3compat.py:

def u(s, encoding='utf8'):
    """cast bytes or unicode to unicode"""
    if isinstance(s, bytes):
        try:
            return s.decode(encoding)
        except UnicodeDecodeError:
            return s.decode('ISO-8859-1')
    elif isinstance(s, str):
        return s
    else:
        raise TypeError("Expected unicode or bytes, got %r" % s)

Upvotes: 2

Related Questions