Neatly pass positional arguments as args and optional arguments as kwargs from argparse to a function

Question

I would like to write a Python script that takes some necessary positional and some optional command-line arguments via argparse:

• Let's call the positional args a,b,c, and the optional arguments x,y,z.
• In my Python script, I would like to pass these args on to a function; specifically, I want a,b,c to be passed as *args, and x,y,z to be passed as **kwargs, the latter retaining their names.
• I would like to do this many times with different functions and different numbers of positional and optional arguments. Is there a neat, flexible, and/or pythonic way to do this?

Here is some example code:

import argparse

def parse():
    parser = argparse.ArgumentParser()
    parser.add_argument('a', help='1st arg')
    parser.add_argument('b', help='2nd arg')
    parser.add_argument('c', help='3rd arg')
    parser.add_argument('-x', '--x', help='1st kwarg')
    parser.add_argument('-y', '--y', help='2nd kwarg')
    parser.add_argument('-z', '--z', help='3rd kwarg')
    return parser.parse_args()

def func(*args, **kwargs):
    a, b, c = args
    print 'a=', a
    print 'b=', b
    print 'c=', c
    for k, v in kwargs.iteritems():
        print '%s=' % k, v

if __name__ == '__main__':

    all_args = parse()
    ### need to split all_args into args and kwargs here ###
    func(*args, **kwargs)

Answer 1

The Namespace you get back from parse_args will have attributes corresponding to each of your arguments. There will be no distinction between the positional arguments and the optionals, e.g.:

args
Namespace(a='1',b='one',x='foo', y=...)

Which, as is well documented, can be accessed as:

args.a
args.x
etc.

The Namespace can also be turned into a dictionary:

vars(args)
{'a'='1', 'b'='one', etc.}

You can pass the dictionary to a function as **kwargs. That's standard Python argument practice.

If you want to pass some arguments as *args you'll have to split those off the Namespace or dictionary yourself. Nothing in argparse will do that for you.

You could write a function like (not tested):

def split_args(args):
   vargs = vars(args)
   alist = ['a','b','c']
   args1 = []
   for a in alist:
        v = vargs.pop(a)
        args1.append(v)
   return args1, vars

Or more compactly, put the pop in a list comprehension:

In [702]: vargs = dict(a=1,b=3,c=4,x=5,y=3,z=3)
In [703]: [vargs.pop(a) for a in ['a','b','c']]
Out[703]: [1, 3, 4]
In [704]: vargs
Out[704]: {'y': 3, 'x': 5, 'z': 3}

In [705]: def foo(*args,**kwargs):
   .....:     print(args)
   .....:     print(kwargs)
   .....:     
In [706]: vargs = dict(a=1,b=3,c=4,x=5,y=3,z=3)
In [707]: foo(*[vargs.pop(a) for a in ['a','b','c']],**vargs)
(1, 3, 4)
{'x': 5, 'z': 3, 'y': 3}

Further idea using custom Action class

The parser determines whether an argument is an optional vs. positional by its option_strings attribute. add_argument returns an Action subclass, which will have attributes like:

MyAction(option_strings=[], dest='baz', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)

This is a positional because option_strings is an empty list.

MyAction(option_strings=['-m', '--mew'], dest='mew', nargs=None,...)

is an optional because that list is not empty.

The parser matches input strings with the option_strings and nargs, and then passes values to the __call__ method of the matching Action. This method is defined like:

def __call__(self, parser, namespace, values, option_string=None):
    setattr(namespace, self.dest, values)

This is the default store action. The values are put in the Namespace as the dest attribute.

The option_string parameter is the string that triggered this call, something like '-m' or '--mew', or None for a positional. The defined action types don't make use of this, but a user-defined Action class could do something.

class MyAction(argparse._StoreAction):
   def __call__(self, parser, namespace, values, option_string=None):
       # store option_string along with values in the Namespace
       setattr(namespace, self.dest, [values,option_string])

Or you could do something special with positionals, e.g.

       if option_string is None:
           # append values to a `star_args` attribute
           # rather than self.dest

With an action like this positionals could be accessed after parsing as:

args.star_args

The parser does maintain a list attribute like this. The extras that parse_known_args returns are stored temporarily in the Namespace in the '_UNRECOGNIZED_ARGS_ATTR' attribute.