Command help (via -h) where `argparse` is range checking input port number

Question

I'm using argparse to parse the inputs to my python3 program. I was recently asked to range check some of the numeric inputs, a seemingly good idea. Argparse has a facility to do just that.

The numeric inputs are port numbers, in the usual range 0-65535, so I altered my parse command line to :

import argparse
cmd_parser = argparse.ArgumentParser()
cmd_parser = add_argument('-p', help='Port number to connect to', dest='cmd_port', default=1234, type=int, choices=range(0,65536))
cmd_parser.parse_args(['-h'])

Now, however, when I request the help, I get flooded with all the possible values from argparse. eg.

optional arguments:
    -h, --help            show this help message and exit
    -p {0,1,2,3,4,5,6,7,8,9,10,11,12,13 ...
    65478,65479,65480,65481,65482,65483,65484,65485,65486,65487,65488,65489,
    65490,65491,65492,65493,65494,65495,65496,65497,65498,65499,65500,65501,
    65502,65503,65504,65505,65506,65507,65508,65509,65510,65511,65512,65513,
    65514,65515,65516,65517,65518,65519,65520,65521,65522,65523,65524,65525,
    65526,65527,65528,65529,65530,65531,65532,65533,65534,65535}
                    Port number to connect to
...

It lists every single port in that range. Is there a way to truncate this or make it realize its a range (0-65535) or for it to use ellipsis or something to make it a bit prettier? Is my only option to explicitly range check my inputs with if statements?

I've been googling this but I'm having trouble finding examples where people used argparse and specified choices. I also checked the documentation on argparse but didn't see anything useful. https://docs.python.org/2/library/argparse.html

Answer 1

The answer I used is inspired by @hpaulj's answer. However, I agree that the problem really lies at the feet of argparse. I found this SO query when my computer hung trying to allocate gigabytes of space just to output the help text.

My issue with hpaulj's 'Range' class is that when using large upper limits, the message still pushes out very large numbers. The following class uses a infinite upper limit by default.

class ArgRange(object):
    from decimal import Decimal
    huge = Decimal('+infinity')
    huge_str = '{:.4E}'.format(huge)

    def __init__(self, start, stop=huge, n=3):
        self.start = start
        self.stop = stop
        self.n = n

    def __contains__(self, key):
        return self.start <= key < self.stop

    def __iter__(self):
        if self.stop < self.start+(self.n*3):
            for i in range(self.start, self.stop):
                yield i
        else:
            for i in range(self.start, self.start+self.n):
                yield I
            if self.stop is self.huge:
                yield '...' + huge_str
            else:
                yield '...'
                for i in range(self.stop - self.n, self.stop):
                    yield i

bounds = ArgRange(2)
balance = ArgRange(0, 1000)
parser = argparse.ArgumentParser(description="Do something fun")
parser.add_argument("width", type=int, choices=bounds,  default=9)
parser.add_argument("height", type=int, choices=balance, default=200)

With an incorrect value, the error is:

argument width: invalid choice: 1 (choose from 2, 3, 4, '...infinity')

or

argument height: invalid choice: 2000 (choose from 0, 1, 2, '...', 997, 998, 999)

And usage looks like:

usage: test.py 
           {2,3,4,...infinity} {0,1,2,...,997,998,999}

Answer 2

There is a Python bug/issue regarding the formatting of large choices lists. Currently choices in the help is formatted with

    def _metavar_formatter: ...
        choice_strs = [str(choice) for choice in action.choices]
        result = '{%s}' % ','.join(choice_strs)

and for errors with:

def _check_value(self, action, value):
    # converted value must be one of the choices (if specified)
    if action.choices is not None and value not in action.choices:
        args = {'value': value,
                'choices': ', '.join(map(repr, action.choices))}
        msg = _('invalid choice: %(value)r (choose from %(choices)s)')
        raise ArgumentError(action, msg % args)

So it expects choices to be an iterable, but does nothing to compress the list or make it pretty. Note that the only test is that value not in action.choices. choices is a very simple feature.

I mentioned this issue in a previous SO question: http://bugs.python.org/issue16468. Proposed patches are involved, so don't expect any fix soon.

I would recommend using your own type testing rather than choices. Or do your own range testing after parsing.

def myrange(astring):
   anint = int(astring)
   if anint in range(0,1000):
        return anint
   else:
        raise ValueError()
        # or for a custom error message
        # raise argparse.ArgumentTypeError('valid range is ...')

parser.add_argument('value',type=myrange,metavar='INT',help='...')

Another old (2012) SO question that addresses the range choices. The answer suggests both a help-formatter fix and the custom type

Python's argparse choices constrained printing

and

Python argparse choices from an infinite set

============================

Out of curiosity I defined a custom range class. It behaves like a regular range (without the step) for the in test, but returns custom values when used as an iterator.

class Range(object):
    def __init__(self, start, stop, n=3):
        self.start = start
        self.stop = stop
        self.n = n
    def __contains__(self, key):
        return self.start<=key<self.stop   
    def __iter__(self):
        if self.stop<(self.start+(self.n*3)):
            for i in range(self.start, self.stop):
                yield i
        else:
            for i in range(self.start, self.start+self.n):
                yield i
            yield '...'
            for i in range(self.stop-self.n, self.stop):
                yield i

When used with an argument as

parser.add_argument("-p",type=int, choices=Range(2,10,2))

It produces

1455:~/mypy$ python stack37680645.py -p 3
Namespace(p=3)

1458:~/mypy$ python stack37680645.py -h
usage: stack37680645.py [-h] [-p {2,3,...,8,9}]

optional arguments:
  -h, --help        show this help message and exit
  -p {2,3,...,8,9}

The error message isn't quite what I'd want, but close

1458:~/mypy$ python stack37680645.py -p 30
usage: stack37680645.py [-h] [-p {2,3,...,8,9}]
stack37680645.py: error: argument -p: invalid choice: 30 (choose from 2, 3, '...', 8, 9)

It would be better if the choices formatting allowed the action.choices object to create its own str or repr string.

Actually the iter could be as simple as (just one string):

def __iter__(self):
    yield 'a custom list'

Another option is to use metavar to control the usage/help display, and this __iter__ to control the error display.

One thing to watch out for when using metavar. The usage formatter does not behave when there are special characters like space, '()' and '[]' in the metavar, especially when the usage line extends to 2 or more lines. That's a known bug/issue.

Answer 3

Use custom action...

import argparse

class PortAction(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        if not 0 < values < 2**16:
            raise argparse.ArgumentError(self, "port numbers must be between 0 and 2**16")
        setattr(namespace, self.dest, values)

cmd_parser = argparse.ArgumentParser()
cmd_parser.add_argument('-p',
                        help='Port number to connect to',
                        dest='cmd_port',
                        default=1234,
                        type=int,
                        action=PortAction,
                        metavar="{0..65535}")

An invalid port number will display the error message based on the raised ArgumentError. If you enter a value of 65536, the following line will be printed:

error: argument -p: port numbers must be between 0 and 2**16

The usage and help messages will be printed based on the metavar displayed

Answer 4

Monkey patch the print_help to get the desired output

def my_help(): print "0-65535 range"
cmd_parser = argparse.ArgumentParser()
cmd_parser.add_argument('-p', help='Port number to connect to',   dest='cmd_port', default=1234, type=int, choices=range(0,65536))
cmd_parser.print_help = my_help
cmd_parser.parse_args()

Answer 5

Supply an explicit metavar argument instead of letting argparse produce one for you.

cmd_parser.add_argument('-p',
                        help='Port number to connect to',
                        dest='cmd_port',
                        default=1234,
                        type=int,
                        choices=range(0,65536),
                        metavar="{0..65535}")

Answer 6

Just use int as type in add_argument, and manually verify it's in the permitted range. Or, use a type of your own, which has a constructor that does the checking for you, and a __int__ method for implicit conversion:

class portnumber:
    def __init__(self, string):
        self._val = int(string)
        if (not self._val > 0) or (not self.val < 2**16):
            raise argparse.ArgumentTypeError("port numbers must be integers between 0 and 2**16")
    def __int__(self):
        return self._val

...

parser.add_argument("-p",type=portnumber)