distutils: How to pass a user defined parameter to setup.py?

Question

How can I pass a user-defined parameter both from the command line and setup.cfg configuration file to distutils' setup.py script?

I want to write a setup.py script, which accepts my package specific parameters. For example:

python setup.py install -foo myfoo

Answer 1

To be fully compatible with both python setup.py install and pip install . you need to use environment variables because pip option --install-option= is bugged:

1. pip --install-option leaks across lines
2. Determine what should be done about --(install|global)-option with Wheels
3. pip not naming abi3 wheels correctly

This is a full example not using the --install-option:

import os
environment_variable_name = 'MY_ENVIRONMENT_VARIABLE'
environment_variable_value = os.environ.get( environment_variable_name, None )

if environment_variable_value is not None:
    sys.stderr.write( "Using '%s=%s' environment variable!\n" % (
            environment_variable_name, environment_variable_value ) )

setup(
        name = 'packagename',
        version = '1.0.0',
        ...
)

Then, you can run it like this on Linux:

MY_ENVIRONMENT_VARIABLE=1 pip install .
MY_ENVIRONMENT_VARIABLE=1 pip install -e .
MY_ENVIRONMENT_VARIABLE=1 python setup.py install
MY_ENVIRONMENT_VARIABLE=1 python setup.py develop

But, if you are on Windows, run it like this:

set "MY_ENVIRONMENT_VARIABLE=1" && pip install .
set "MY_ENVIRONMENT_VARIABLE=1" && pip install -e .
set "MY_ENVIRONMENT_VARIABLE=1" && python setup.py install
set "MY_ENVIRONMENT_VARIABLE=1" && python setup.py develop

References:

1. How to obtain arguments passed to setup.py from pip with '--install-option'?
2. Passing command line arguments to pip install
3. Passing the library path as a command line argument to setup.py

Answer 2

Yes, it's 2015 and the documentation for adding commands and options in both setuptools and distutils is still largely missing.

After a few frustrating hours I figured out the following code for adding a custom option to the install command of setup.py:

from setuptools.command.install import install


class InstallCommand(install):
    user_options = install.user_options + [
        ('custom_option=', None, 'Path to something')
    ]

    def initialize_options(self):
        install.initialize_options(self)
        self.custom_option = None

    def finalize_options(self):
        #print('The custom option for install is ', self.custom_option)
        install.finalize_options(self)

    def run(self):
        global my_custom_option
        my_custom_option = self.custom_option
        install.run(self)  # OR: install.do_egg_install(self)

It's worth to mention that install.run() checks if it's called "natively" or had been patched:

if not self._called_from_setup(inspect.currentframe()):
    orig.install.run(self)
else:
    self.do_egg_install()

At this point you register your command with setup:

setup(
    cmdclass={
        'install': InstallCommand,
    },
    :

Answer 3

Perhaps you are an unseasoned programmer like me that still struggled after reading all the answers above. Thus, you might find another example potentially helpful (and to address the comments in previous answers about entering the command line arguments):

class RunClientCommand(Command):
    """
    A command class to runs the client GUI.
    """

    description = "runs client gui"

    # The format is (long option, short option, description).
    user_options = [
        ('socket=', None, 'The socket of the server to connect (e.g. '127.0.0.1:8000')',
    ]

    def initialize_options(self):
        """
        Sets the default value for the server socket.

        The method is responsible for setting default values for
        all the options that the command supports.

        Option dependencies should not be set here.
        """
        self.socket = '127.0.0.1:8000'

    def finalize_options(self):
        """
        Overriding a required abstract method.

        The method is responsible for setting and checking the 
        final values and option dependencies for all the options 
        just before the method run is executed.

        In practice, this is where the values are assigned and verified.
        """
        pass

    def run(self):
        """
        Semantically, runs 'python src/client/view.py SERVER_SOCKET' on the
        command line.
        """
        print(self.socket)
        errno = subprocess.call([sys.executable, 'src/client/view.py ' + self.socket])
        if errno != 0:
            raise SystemExit("Unable to run client GUI!")

---

setup(
    # Some other omitted details
    cmdclass={
        'runClient': RunClientCommand,
    },

The above is tested and from some code I wrote. I have also included slightly more detailed docstrings to make things easier to understand.

As for the command line: python setup.py runClient --socket=127.0.0.1:7777. A quick double check using print statements shows that indeed the correct argument is picked up by the run method.

Other resources I found useful (more and more examples):

Custom distutils commands

https://seasonofcode.com/posts/how-to-add-custom-build-steps-and-commands-to-setuppy.html

Answer 4

I successfully used a workaround to use a solution similar to totaam's suggestion. I ended up popping my extra arguments from the sys.argv list:

import sys
from distutils.core import setup
foo = 0
if '--foo' in sys.argv:
    index = sys.argv.index('--foo')
    sys.argv.pop(index)  # Removes the '--foo'
    foo = sys.argv.pop(index)  # Returns the element after the '--foo'
# The foo is now ready to use for the setup
setup(...)

Some extra validation could be added to ensure the inputs are good, but this is how I did it

Answer 5

As Setuptools/Distuils are horribly documented, I had problems finding the answer to this myself. But eventually I stumbled across this example. Also, this similar question was helpful. Basically, a custom command with an option would look like:

from distutils.core import setup, Command

class InstallCommand(Command):
    description = "Installs the foo."
    user_options = [
        ('foo=', None, 'Specify the foo to bar.'),
    ]
    def initialize_options(self):
        self.foo = None
    def finalize_options(self):
        assert self.foo in (None, 'myFoo', 'myFoo2'), 'Invalid foo!'
    def run(self):
        install_all_the_things()

setup(
    ...,
    cmdclass={
        'install': InstallCommand,
    }
)

Answer 6

Here is a very simple solution, all you have to do is filter out sys.argv and handle it yourself before you call to distutils setup(..). Something like this:

if "--foo" in sys.argv:
    do_foo_stuff()
    sys.argv.remove("--foo")
...
setup(..)

The documentation on how to do this with distutils is terrible, eventually I came across this one: the hitchhikers guide to packaging, which uses sdist and its user_options. I find the extending distutils reference not particularly helpful.

Although this looks like the "proper" way of doing it with distutils (at least the only one that I could find that is vaguely documented). I could not find anything on --with and --without switches mentioned in the other answer.

The problem with this distutils solution is that it is just way too involved for what I am looking for (which may also be the case for you). Adding dozens of lines and subclassing sdist is just wrong for me.

Answer 7

A quick and easy way similar to that given by totaam would be to use argparse to grab the -foo argument and leave the remaining arguments for the call to distutils.setup(). Using argparse for this would be better than iterating through sys.argv manually imho. For instance, add this at the beginning of your setup.py:

argparser = argparse.ArgumentParser(add_help=False)
argparser.add_argument('--foo', help='required foo argument', required=True)
args, unknown = argparser.parse_known_args()
sys.argv = [sys.argv[0]] + unknown

The add_help=False argument means that you can still get the regular setup.py help using -h (provided --foo is given).

Answer 8

You can't really pass custom parameters to the script. However the following things are possible and could solve your problem:

• optional features can be enabled using --with-featurename, standard features can be disabled using --without-featurename. [AFAIR this requires setuptools]
• you can use environment variables, these however require to be set on windows whereas prefixing them works on linux/ OS X (FOO=bar python setup.py).
• you can extend distutils with your own cmd_classes which can implement new features. They are also chainable, so you can use that to change variables in your script. (python setup.py foo install) will execute the foo command before it executes install.

Hope that helps somehow. Generally speaking I would suggest providing a bit more information what exactly your extra parameter should do, maybe there is a better solution available.