Argparse default as function is always called

Question

//edit 1: Changed slightly get_variable - forgot to add another argument that is passed to it (was writing it from memory, sorry for that).

I have a problem with default values from argparser.

Some values if not present in command line, are taken from environment using os.env, and if there is none, get it from DEFAULT_FOR_VARIABLE:

def get_variable(name, DEFAULT_FOR_VARIABLE = ''):
    if name in os.environ:
        return os.environ[name]
    print("no default value")
    return DEFAULT_FOR_VARIABLE

This is how it's parsed in main():

parser = argparse.ArgumentParser(description=MODULE_NAME)
parser.add_argument('--test_arg', default=get_variable(VAR_NAME, DEFAULT_FOR_TEST_ARG))
args = parser.parse_args()
print(args.test_arg)

No matter if I pass down arguments or not, get_variable function is called and if there is no value in os.environ, print gets executed (to let me know there is missing argument), even when there is a value passed:

λ python Parser_Test.py --test_arg test_arg
no default value
test_arg

It's working as expected when arguments are not passed:

λ python Parser_Test.py
No default value

But when for DEFAULT_FOR_TEST_ARG is set:

λ python Parser_Test.py
  No default value
  DEFAULT_VALUE_FOR_TEST_ARG

Also checking each parsed argument would be hard, since there is no way of iterating them the way it's provided by argparse - I have quite few of them to check for from the user.

Is there a way to change this behavior? Or should I use non-standard module for parsing arguments in such a case?

Answer 1

For a one-liner you could also try this:

os.environ.get(VAR_NAME, DEFAULT_FOR_TEST_ARG)

Answer 2

Here is an approach using a lambda as the default in python 3.6. I think this is on target with what the OP wanted to do. The default doesn't get evaluated immediately. You can easily find them and call them in a for loop to resolve the values. I included the t2 argument with a string default just to show that a normal default still works fine in this context.

import argparse
import os


def get_value(var, dflt):
    if var in os.environ:
        return os.environ[var]
    return dflt


parser = argparse.ArgumentParser(description=os.path.splitext(os.path.basename(__file__))[0])
parser.add_argument('--t1', default=lambda: get_value('t1_value', 't1 default'))
parser.add_argument('--t2', default='t2 default')
args = parser.parse_args()
print("Arguments have been parsed")
print(f"--t1: {args.t1}")
print(f"--t2: {args.t2}")


print("Lazily getting defaults")
for key in vars(args):
    f = args.__dict__[key]
    if callable(f):
        print(f'Getting default value for {key}')
        args.__dict__[key] = f()

print(f"--t1: {args.t1}")
print(f"--t2: {args.t2}")

Results:

Connected to pydev debugger (build 202.7660.27)
Arguments have been parsed
--t1: <function <lambda> at 0x000002425DD3FAE8>
--t2: t2 default
Lazily getting defaults
Getting default value for t1
--t1: t1_default
--t2: t2 default

Process finished with exit code 0

You can do a similar thing with a specialized class, but I think the lambda is more concise and essentially the same.

Answer 3

get_variable(VAR_NAME) is evaluated by the interpreter when the add_argument method is used. In python function arguments are evaluated before being passed to the function.

argparse does defer evaluating the default if it is a string:

In [271]: p = argparse.ArgumentParser()
In [272]: p.add_argument('-f', type=int, default='12');
In [273]: p.parse_args('-f 23'.split())
Out[273]: Namespace(f=23)
In [274]: p.parse_args([])
Out[274]: Namespace(f=12)

Here, if no -f is provided, the '12' will be passed to the type function:

int('12')

or with a custom type:

In [275]: def mytype(astr):
     ...:     print('eval',astr)
     ...:     return int(astr)
In [276]: p.add_argument('-g', type=mytype, default='12');
In [277]: p.parse_args([])
eval 12
Out[277]: Namespace(f=12, g=12)
In [278]: p.parse_args(['-g','3'])
eval 3
Out[278]: Namespace(f=12, g=3)

But in your case the code that you want to conditionally evaluate probably can't be handled by a type function. That is, you aren't evaluating the default in the same way as you would an user provided string.

So a post parsing test probably makes most sense. The default default is None, which is easily tested:

if args.test is None:
     args.test = 'the proper default'

The user can't provide any string that will produce None, so it is a safe default.

---

Just out of curiosity I wrote a type that looks up a name in os.environ:

In [282]: def get_environ(name):
     ...:     if name in os.environ:
     ...:         return os.environ[name]
     ...:     raise argparse.ArgumentTypeError('%s not in environ'%name)

In [283]: p.add_argument('-e', type=get_environ, default='DISPLAY');

Without arguments it looks up the default os.environ['DISPLAY']

In [284]: p.parse_args([])
eval 12
Out[284]: Namespace(e=':0', f=12, g=12)

with a valid name:

In [289]: p.parse_args(['-e','EDITOR'])
eval 12
Out[289]: Namespace(e='nano', f=12, g=12)

and raises an error when the name isn't valid:

In [290]: p.parse_args(['-e','FOO'])
usage: ipython3 [-h] [-f F] [-g G] [-e E]
ipython3: error: argument -e: FOO not in environ
An exception has occurred, use %tb to see the full traceback.

I know it's not what you are aiming for, but it gives an idea of what is possible if you want to delay evaluation of a default.

Answer 4

Not sure if I fully understand, but can you not do this?

def get_variable(name):
    if name in os.environ:
        return os.environ[name]
    else:
        print("no default value")
        return 'empty'

Or:

parser = argparse.ArgumentParser(description=MODULE_NAME)
parser.add_argument('--test_arg',dest='test',nargs='?', default="empty")
args = parser.parse_args()
if args.test == "empty":
    if name in os.environ:
        newGlobalVar = os.environ["name"]
        print("no default value")
    else:
        newGlobalVar = args.test