Python Argument Parser, Raising an exception before -h

Question

I don't know why this is happening. It was my understanding that the user would at least get a chance to use -h before the default actions were executed.

import os, sys, argparse
class argument_parser():
  # call on all our file type parsers in the sequence_anlysis_method

    def __init__(self):
        self.db_directory = os.path.dirname(os.path.abspath(sys.argv[0]))

        """A customized argument parser that does a LOT of error checking"""
        self.parser = argparse.ArgumentParser(
            prog="igblast")

        general = self.parser.add_argument_group(
            title="\nGeneral Settings")

        general.add_argument(
            "-x", '--executable',
            default="/usr/bin/igblastn",
            type=self._check_if_executable_exists,
            help="The location of the executable, default is /usr/bin/igblastn")

        self.args = self.parser.parse_args()

    def _check_if_executable_exists(self,x_path):
        if not os.path.exists(x_path):
            msg = "path to executable {0} does not exist, use -h for help\n".format(x_path)
            raise argparse.ArgumentTypeError(msg)
        if not os.access(x_path, os.R_OK):
            msg1 = "executable {0} does have not permission to run\n".format(x_path)
            raise argparse.ArgumentTypeError(msg1)
        else:
            return x_path

if __name__ == '__main__':
    argument_class = argument_parser()

Now if /usr/bin/igblastn is there, then it's fine but if its not, just calling on this program raises the exception in self_check_if_executable_exists.

File "execution.py", line 220, in <module>
    argument_class = ap()
  File "/home/willisjr/utilities/pyig/src/arg_parse.py", line 159, in __init__
    self.args = self.parser.parse_args()
  File "/usr/local/lib/python2.7/argparse.py", line 1656, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/local/lib/python2.7/argparse.py", line 1678, in parse_known_args
    default = self._get_value(action, default)
  File "/usr/local/lib/python2.7/argparse.py", line 2203, in _get_value
    raise ArgumentError(action, msg)
argparse.ArgumentError: argument -x/--executable: path to executable /usr/bin/igblastn does not exist, use -h for help

It was my understanding that the user always had a chance to run --help or -h or before any action was taken resulting in this argument error. Is my understanding of the arg parser not clear?

Answer 1

In Python 2.7.3, early in parse_known_args (called by parse_args), default values are inserted in the namespace.

    # add any action defaults that aren't present
    for action in self._actions:
        if action.dest is not SUPPRESS:
            if not hasattr(namespace, action.dest):
                if action.default is not SUPPRESS:
                    default = action.default
                    if isinstance(action.default, basestring):  # delayed in 3.3.1
                        default = self._get_value(action, default)
                    setattr(namespace, action.dest, default)

If the default is a string, it is passed through _get_value which calls the appropriate type, in your case _check_if_executable_exists. That produces the error that you see.

In the News for 3.3.1 https://docs.python.org/3.3/whatsnew/changelog.html

Issue #12776, issue #11839: Call argparse type function (specified by add_argument) only once. Before, the type function was called twice in the case where the default was specified and the argument was given as well. This was especially problematic for the FileType type, as a default file would always be opened, even if a file argument was specified on the command line.

With this change, the _get_value call is postponed to the end of parse_known_args, and is called only if parsing has not put some other value there (the default is needed).

    if isinstance(action.default, basestring):  # dropped in 3.3.1
        default = self._get_value(action, default)

So your script runs as expected (with '-h') on my development copy. I'm not entirely sure which versions of Python have this correction.

So until you can run a newer Python version, it's your responsibility to ensure that the default is a valid value. Even with this bug fix, it's a good idea make sure that your defaults are valid before you give them to the add_argument() call. An invalid default will confuse your user regardless of when it is handled.

Answer 2

You forgot to run parse_args() at the end of init __init__

class argument_parser(object):
    # call on all our file type parsers in the sequence_anlysis_method

    def __init__(self):
        self.db_directory = os.path.dirname(os.path.abspath(sys.argv[0]))

        """A customized argument parser that does a LOT of error checking"""
        self.parser = argparse.ArgumentParser(
            prog="igblast")

        general = self.parser.add_argument_group(
            title="\nGeneral Settings")

        general.add_argument(
            "-x", '--executable',
            default="/usr/bin/igblastn",
            type=self._check_if_executable_exists,
            help="The location of the executable, default is /usr/bin/igblastn")


        """ >>>>>>>>>>>>> add this line  <<<<<<<<<<< """
        self.parser.parse_args()

    def _check_if_executable_exists(self,x_path):
        if not os.path.exists(x_path):
            msg = "path to executable {0} does not exist, use -h for help\n".format(x_path)
            raise argparse.ArgumentTypeError(msg)
        if not os.access(x_path, os.R_OK):
            msg1 = "executable {0} does have not permission to run\n".format(x_path)
            raise argparse.ArgumentTypeError(msg1)
        else:
            return x_path

if __name__ == '__main__':
    argument_class = argument_parser()