Implement an interactive shell over ssh in Python using Paramiko?

Question

I want to write a program (in Python 3.x on Windows 7) that executes multiple commands on a remote shell via ssh. After looking at paramikos' exec_command() function, I realized it's not suitable for my use case (because the channel gets closed after the command is executed), as the commands depend on environment variables (set by prior commands) and can't be concatenated into one exec_command() call as they are to be executed at different times in the program.

Thus, I want to execute commands in the same channel. The next option I looked into was implementing an interactive shell using paramikos' invoke_shell() function:

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(host, username=user, password=psw, port=22)

channel = ssh.invoke_shell()

out = channel.recv(9999)

channel.send('cd mivne_final\n')
channel.send('ls\n')

while not channel.recv_ready():
    time.sleep(3)

out = channel.recv(9999)
print(out.decode("ascii"))

channel.send('cd ..\n')
channel.send('cd or_fail\n')
channel.send('ls\n')

while not channel.recv_ready():
    time.sleep(3)

out = channel.recv(9999)
print(out.decode("ascii"))

channel.send('cd ..\n')
channel.send('cd simulator\n')
channel.send('ls\n')

while not channel.recv_ready():
    time.sleep(3)

out = channel.recv(9999)
print(out.decode("ascii"))

ssh.close() 

There are some problems with this code:

1. The first print doesn't always print the ls output (sometimes it is only printed on the second print).
2. The first cd and ls commands are always present in the output (I get them via the recv command, as part of the output), while all the following cd and ls commands are printed sometimes, and sometimes they aren't.
3. The second and third cd and ls commands (when printed) always appear before the first ls output.

I'm confused with this "non-determinism" and would very much appreciate your help.

Answer 1

I needed this for a Cisco router, which is rather different from a Linux machine. Credit to @misha for identifying the big challenge when running multiple commands during a single session, namely detecting the end of one command output so you know when to stop reading from the router's stdout object. If you don't detect that, the read loop just hangs. Sending two commands every time and using the second command as a sentinel is kind of a clever hack, so I copied it! This uses a well-known error response from the IOS command prompt as a sentinel.

import logging
import paramiko
import socket

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)

def inter_handler(
        title: str,
        instructions: str,
        prompts: List[Tuple[str, bool]]) -> List[str]:
    """
    Proprietary details omitted. Here is the relevant doc from
    paramiko/transport.py, function `auth_interactive`:

        The handler is expected to be a callable that will handle calls of the
        form: ``handler(title, instructions, prompt_list)``.  The ``title`` is
        meant to be a dialog-window title, and the ``instructions`` are user
        instructions (both are strings).  ``prompt_list`` will be a list of
        prompts, each prompt being a tuple of ``(str, bool)``.  The string is
        the prompt and the boolean indicates whether the user text should be
        echoed.

        A sample call would thus be:
        ``handler('title', 'instructions', [('Password:', False)])``.

        The handler should return a list or tuple of answers to the server's
        questions.
    """
    pass


def ssh_run_cmds(
        host: str,
        port: int,
        user: str,
        commands: list) -> None:
    """
    Connect to the router, authenticate by computing a challenge
    response, and run commands. A major challenge is detecting
    the end of the command output, to know when to stop reading
    from the router session. This code uses an ugly hack of
    sending an invalid command and checking for a well-known
    error message.
    """
    # Create a socket and connect it to port 22 on the remote host
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    # the argument must be a tuple
    sock.connect((host, port))
    # Wrap the socket in a paramiko Transport object
    ts = paramiko.Transport(sock)
    # Tell Paramiko that the Transport is going to be used as a client
    ts.start_client(timeout=10)
    # Authenticate the specified user via the handler
    ts.auth_interactive(user, inter_handler)
    # Open a channel
    chan = ts.open_channel(kind='session', timeout=10)
    # Associate a pseudo tty
    chan.get_pty()
    # Request an interactive shell session
    chan.invoke_shell()
    # Create writer/reader file-like objects
    stdin = chan.makefile('wb')
    stdout = chan.makefile('r')
    # Use the output from this invalid command as a sentinel
    bogus_cmd = 'show bogus'
    for cmd in commands:
        # Send the command AND a bogus command to detect end of output
        cmds = f'{cmd}\n{bogus_cmd}\n'
        logger.debug('Send commands: %s', cmds)
        stdin.write(cmds)
        stdin.flush()

        # Read the response
        for line in stdout:
            line = line.strip()
            logger.debug('Output line: %s', line)
            # the response from the bogus command is the last line
            if line.startswith("% Invalid input detected at '^' marker."):
                break
        # for line
    # for cmd

    stdin.close()
    stdout.close()
    chan.close()
    ts.close()
    sock.close()

Answer 2

I tried the answer above, and it didn't work because ECHO command returned error in Python CLI, which I was using with SSH.

So I wrote another code that is applicable for Python CLI, assuming that the output is in one line.

And I also think something like f"print('{finish}')" can do same thing (terminator??) as ECHO in the answer above. But I didn't make use of it because my output always has to be in one line.

class MusicPlayer:
def __init__(self, host='', username='pi', password=''):
    self.ssh = paramiko.SSHClient()
    self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    self.ssh.connect(host, username=username, password=password)
    channel = self.ssh.invoke_shell()
    self.stdin = channel.makefile('wb')
    self.stdout = channel.makefile('r')
    self.in_history = []
    self.out_history = []
    self.init_vlc()
    self.print()
    # atexit.register(self.__del__)

def __del__(self):
    self.ssh.close()

def execute(self, cmd):
    self.in_history.append(cmd)
    self.stdin.write(cmd + '\n')

def print(self, lines=1):
    for line in self.stdout:
        lined = line.strip()
        print(lined)
        self.out_history.append(lined)
        if self.in_history[-1] in lined:
            next_one = self.stdout.__next__().strip()
            print(next_one)
            self.out_history.append(next_one)
            return next_one

def init_vlc(self):
    for command in ['python', 'import vlc', 'import time', 'media_player = vlc.MediaPlayer()']:
        self.execute(command)

Answer 3

import paramiko
import re


class ShellHandler:

    def __init__(self, host, user, psw):
        self.ssh = paramiko.SSHClient()
        self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        self.ssh.connect(host, username=user, password=psw, port=22)

        channel = self.ssh.invoke_shell()
        self.stdin = channel.makefile('wb')
        self.stdout = channel.makefile('r')

    def __del__(self):
        self.ssh.close()

    def execute(self, cmd):
        """

        :param cmd: the command to be executed on the remote computer
        :examples:  execute('ls')
                    execute('finger')
                    execute('cd folder_name')
        """
        cmd = cmd.strip('\n')
        self.stdin.write(cmd + '\n')
        finish = 'end of stdOUT buffer. finished with exit status'
        echo_cmd = 'echo {} $?'.format(finish)
        self.stdin.write(echo_cmd + '\n')
        shin = self.stdin
        self.stdin.flush()

        shout = []
        sherr = []
        exit_status = 0
        for line in self.stdout:
            if str(line).startswith(cmd) or str(line).startswith(echo_cmd):
                # up for now filled with shell junk from stdin
                shout = []
            elif str(line).startswith(finish):
                # our finish command ends with the exit status
                exit_status = int(str(line).rsplit(maxsplit=1)[1])
                if exit_status:
                    # stderr is combined with stdout.
                    # thus, swap sherr with shout in a case of failure.
                    sherr = shout
                    shout = []
                break
            else:
                # get rid of 'coloring and formatting' special characters
                shout.append(re.compile(r'(\x9B|\x1B\[)[0-?]*[ -/]*[@-~]').sub('', line).
                             replace('\b', '').replace('\r', ''))

        # first and last lines of shout/sherr contain a prompt
        if shout and echo_cmd in shout[-1]:
            shout.pop()
        if shout and cmd in shout[0]:
            shout.pop(0)
        if sherr and echo_cmd in sherr[-1]:
            sherr.pop()
        if sherr and cmd in sherr[0]:
            sherr.pop(0)

        return shin, shout, sherr