filename tab-completion in Cmd.cmd of Python

Question

I'm working on making a command-line tool using Cmd.cmd of Python, and I want to add a "load" command with filename argument, which is supporting tab-completion.

Referring this and this, I mad a code like this:

import os, cmd, sys, yaml
import os.path as op
import glob as gb

def _complete_path(path):
    if op.isdir(path):
        return gb.glob(op.join(path, '*'))
    else:
        return gb.glob(path+'*')

class CmdHandler(cmd.Cmd):

    def do_load(self, filename):
        try:
            with open(filename, 'r') as f:
                self.cfg = yaml.load(f)
        except:
            print 'fail to load the file "{:}"'.format(filename)

    def complete_load(self, text, line, start_idx, end_idx):
        return _complete_path(text)

This works well for cwd, however, when I want to go into subdir, after subdir/ then the "text" of complete_load function becomes blank, so _complete_path func returns cwd again.

I don't know how to get the contents of subdir with tab-completion. Please help!

Answer 1

This works for me. Remove the "self" if you are not using within a class.

def _complete_path(self, path):
    if os.path.isdir(path):
        return gb.glob(os.path.join(path, '*'))
    else:
        return gb.glob(path + '*')

def complete_load(self, text, line, start_idx, end_idx):
    mline = line.split(' ')[-1]
    offs = len(mline) - len(text)
    completions = []
    if line.split()[-2] == '-p':
        completions = self._complete_path(mline)
    return [s[offs:] for s in completions if s.startswith(mline)]

Answer 2

I use shlex to parse the line. In contrast to some other solutions I support quoted and escaped paths (i.e. paths with whitespace) and completion works for any cursor position. I did not test extensively so your mileage may vary.

def path_completion(self, text, line, startidx, endidx):
    try:
        glob_prefix = line[:endidx]

        # add a closing quote if necessary
        quote = ['', '"', "'"]
        while len(quote) > 0:
            try:
                split = [s for s in shlex.split(glob_prefix + quote[0]) if s.strip()]
            except ValueError as ex:
                assert str(ex) == 'No closing quotation', 'Unexpected shlex error'
                quote = quote[1:]
            else:
                break
        assert len(quote) > 0, 'Could not find closing quotation'

        # select relevant line segment
        glob_prefix = split[-1] if len(split) > 1 else ''

        # expand tilde
        glob_prefix = os.path.expanduser(glob_prefix)

        # find matches
        matches = glob.glob(glob_prefix + '*')

        # append os.sep to directories
        matches = [match + os.sep if Path(match).is_dir() else match for match in matches]

        # cutoff prefixes
        cutoff_idx = len(glob_prefix) - len(text)
        matches = [match[cutoff_idx:] for match in matches]

        return matches
    except:
        traceback.print_exc()

Answer 3

I accomplished this by doing:

def complete_listFolder(self, text, line, begidx, endidx):
    path = os.path.relpath(os.path.normpath(line.split()[1]))
            if not os.path.isdir(path) and not os.path.isfile(path):
                baseName = os.path.basename(path)
                dirName = os.path.dirname(path)
                return fnmatch.filter(os.listdir(dirName), baseName + "*")

            completions = [completion for completion in os.listdir(path)]    
            return completions

Off course there is alot to improve but hope this helps.

=)

Answer 4

Your primary issue is that the readline library is delimiting things based on it's default delimiter set:

import readline
readline.get_completer_delims()
# yields ' \t\n`~!@#$%^&*()-=+[{]}\\|;:\'",<>/?'

When tab completing for a file name I remove everything from this but whitespace.

import readline
readline.set_completer_delims(' \t\n')

After setting the delimiters, the 'text' parameter to your completion function should be more what you are expecting.

This also resolves commonly encountered issues with tab completion duplicating part of your text.

Answer 5

Implementing filename completion with cmd is a bit tricky because the underlying readline library interprets special characters such as '/' and '-' (and others) as separators, and this sets which substring within the line is to be replaced by the completions.

For example,

> load /hom<tab>

calls complete_load() with

text='hom', line='load /hom', begidx=6, endidx=9
text is line[begidx:endidx]

'text' is not "/hom" because the readline library parsed the line and returns the string after the '/' separator. The complete_load() should return a list of completion strings that begin with "hom", not "/hom", since the completions will replace the substring starting at the begidx. If complete_load() function incorrectly returns ['/home'], the line becomes,

> load //home

which is not good.

Other characters are considered to be separators by readline, not just slashes, so you cannot assume the substring before 'text' is a parent directory. For example:

> load /home/mike/my-file<tab>

calls complete_load() with

text='file', line='load /home/mike/my-file', begidx=19, endidx=23

Assuming /home/mike contains the files my-file1 and my-file2, the completions should be ['file1', 'file2'], not ['my-file1', 'my-file2'], nor ['/home/mike/my-file1', '/home/mike/my-file2']. If you return the full paths, the result is:

> load /home/mike/my-file/home/mike/my-file1

The approach I took was to use the glob module to find the full paths. Glob works for absolute paths and relative paths. After finding the paths, I remove the "fixed" portion, which is the substring before the begidx.

First, parse the fixed portion argument, which is the substring between the space and the begidx.

index = line.rindex(' ', 0, begidx)  # -1 if not found
fixed = line[index + 1: begidx]

The argument is between the space and the end of the line. Append a star to make a glob search pattern.

I append a '/' to results which are directories, as this makes it easier to traverse directories with tab completion (otherwise you need to hit the tab key twice for each directory), and it makes it obvious to the user which completion items are directories and which are files.

Finally remove the "fixed" portion of the paths, so readline will replace just the "text" part.

import os
import glob
import cmd

def _append_slash_if_dir(p):
    if p and os.path.isdir(p) and p[-1] != os.sep:
        return p + os.sep
    else:
        return p

class MyShell(cmd.Cmd):
    prompt = "> "

    def do_quit(self, line):
        return True

    def do_load(self, line):
        print("load " + line)

    def complete_load(self, text, line, begidx, endidx):
        before_arg = line.rfind(" ", 0, begidx)
        if before_arg == -1:
            return # arg not found

        fixed = line[before_arg+1:begidx]  # fixed portion of the arg
        arg = line[before_arg+1:endidx]
        pattern = arg + '*'

        completions = []
        for path in glob.glob(pattern):
            path = _append_slash_if_dir(path)
            completions.append(path.replace(fixed, "", 1))
        return completions

MyShell().cmdloop()

Answer 6

I have same idea with jinserk but in different way. Here's my code:

def complete_load(self, text, line, begidx, endidx):
    arg = line.split()[1:]

    if not arg:
        completions = os.listdir('./')
    else:
        dir, part, base = arg[-1].rpartition('/')
        if part == '':
            dir = './'
        elif dir == '':
            dir = '/'            

        completions = []
        for f in os.listdir(dir):
            if f.startswith(base):
                if os.path.isfile(os.path.join(dir,f)):
                    completions.append(f)
                else:
                    completions.append(f+'/')

    return completions

please let me know if you have better idea. note: I think this method only works on Unix family OS because I create this code based on Unix directory structure.

Answer 7

I don't think this is the best answer, but I got the function what I intend to like this:

def _complete_path(text, line):
    arg = line.split()[1:]
    dir, base = '', ''
    try: 
        dir, base = op.split(arg[-1])
    except:
        pass
    cwd = os.getcwd()
    try: 
        os.chdir(dir)
    except:
        pass
    ret = [f+os.sep if op.isdir(f) else f for f in os.listdir('.') if f.startswith(base)]
    if base == '' or base == '.': 
        ret.extend(['./', '../'])
    elif base == '..':
        ret.append('../')
    os.chdir(cwd)
    return ret

    .............................

    def complete_load(self, text, line, start_idx, end_idx):
        return _complete_path(text, line)

I didn't use "text" from the complete_cmd(), but use a parsing "line" argument directly. If you have any better idea, please let me know.