CODEWITHSUNDEEP
pythonargparse
Dig_Doug
Dig_Doug

Reputation: 233

Why does argparse include default value for optional argument even when argument is specified?

I'm using argparse with Python 3.6. I am using optional arguments to collect my program parameters. For some of these, I have reasonable default values so I configure the parser with the default for that argument.

In [2]: import argparse
   ...: import shlex
   ...: 
   ...: parser = argparse.ArgumentParser()
   ...: parser.add_argument('-s', '--samples', action='store', nargs='+', type=int, required=True,
   ...:                     help='number of samples')
   ...: parser.add_argument('-r', '--regions', action='append', nargs='+', type=str, default=['all'],
   ...:                     help='one or more region names. default to [\'all\']')

When not specifying the -r/--regions argument, I expect to see the configured default value (and I do).

In [3]: s = '-s 37'
   ...: parser.parse_args(shlex.split(s))
Out[3]: Namespace(regions=['all'], samples=[37])

When specifying the -r/--regions argument, I expect to see only the values I provide with the argument, but the default shows up as well.

In [5]: s = '-s 37 -r foo'
...: parser.parse_args(shlex.split(s))
Out[5]: Namespace(regions=['all', ['foo']], samples=[37])

This is not what I expected. I would expect the default to be present only when the optional argument is not present. I stepped through the argparse code. I can't find where the default value is included. Based on the comments, it appears the logic is to add the default value to the resulting Namespace before processing the actual argument value supplied. I would have expected it to be the reverse (i.e. apply the default only when you get to the end of the args and have not seen an argument which has a default.

Can anyone shed light on this? Am I incorrectly using or understanding the purpose of the default option on an optional argument? Is there a way to achieve the behavior I'm seeking (i.e. if the optional is not provided, use the default value in the Namespace)?

Upvotes: 14

Views: 6727

Answers (3)

Sadak
Sadak

Reputation: 911

you simply remove nargs argument and use action=append

parser = argparse.ArgumentParser()
parser.add_argument('-i', default=[], action='append', type=str)
parser.parse_args('-i foo'.split())
# result >> Namespace(i=['foo'])

for my case I didn't want to check if the argument is None and then check the length.

Upvotes: 0

hpaulj
hpaulj

Reputation: 231738

The logic for handling defaults is to insert all defaults into the namespace at the start of parsing. Then let parsing replace them. Then at the end of parsing there's a complicated piece of logic:

for each value in the namespace
   if it is a string and equals the default
      evaluate the string (with `type`) and put it back

For ordinary store actions this works fine, and lets you provide defaults as strings or any value of your choice.

With append this produces your unexpected value. It puts the ['all'] on the namespace, and then appends the new values to that. Since your nargs is '+', it appends a list, resulting in that mix of string and lists.

The append action cannot tell whether it is appending the new value to a list provided by default or a list that's the result of several previous appends. With a None default it will create an empty list, and append to that.

While this doesn't perform as you expected, it actually gives you a lot of control.

The simplest way around this is to leave the default as None. After parsing, just check if this attribute is None, and if so, replace it with your ['all']. It is not evil or contrary to the design intentions of argparse developers. Somethings are easier after all the input has been parsed.

The issue has been raised on Python bug/issues, http://bugs.python.org/issue16399, and probably here on SO before. But I suspect the best a patch can do is to add a note to the documentation, similar to this one from optparse:

"The append action calls the append method on the current value of the option. This means that any default value specified must have an append method. It also means that if the default value is non-empty, the default elements will be present in the parsed value for the option, with any values from the command line appended after those default values".

See the bug/issue for ideas about writing your own append Action subclass.

Upvotes: 11

danny
danny

Reputation: 5270

You are correct that with an append action and a non-empty default, any provided values are appended to the default rather than replacing it - that is the expected behaviour of append.

For the append action, more appropriate code is as follows

parser.add_argument('-r', '--regions', action='append', type=str)
parser.parse_args('-r foo -r foo2'.split())
Namespace(regions=['foo', 'foo2'])

You will note in the original code with nargs='+' the resulting regions value is a list of lists. Append action already makes the variable a list so nargs is not needed.

To then provide a default value that gets overridden by parser make the default be outside the parser's name space, for example

_DEFAULT_REGIONS = ['all']

parser = argparse.ArgumentParser()
parser.add_argument('-r', '--regions', action='append', type=str,
                    help="Defaults to %s" % (_DEFAULT_REGIONS))
parser.parse_args(<..>)

regions = parser.regions \
    if parser.regions is not None else _DEFAULT_REGIONS
function_using_regions(regions)

Uses parser.regions if provided, _DEFAULT_REGIONS otherwise.

Upvotes: 3

Related Questions