How to permit None or a specific type as an argument to a Python C Extension function?

Question

Say I have a contrived function like the following:

static int foo(PyObject *self, PyObject *args) {
    char *a = "";
    char *b = "";
    int c = 0;

    if (!PyArg_ParseTuple(args, "ss|i", &a, &b, &c) {
        return NULL;
    }

    printf("c is %i
", c);

    //some_function_requiring_int_data_type(c);
}

I would like the user to be able to submit either an int or a None as the value to the c arg, but the above code doesn't allow this:

>>>from spam import foo
>>>foo('a', 'b')
c is 0
>>>foo('a', 'b', 100)
c is 100
>>>foo('a', 'b', None)
TypeError: an integer is required

Currently, to enable this behavior I have a bunch of ugly code that looks like the following:

static int foo(PyObject *self, PyObject *args) {
    char *a = "";
    char *b = "";
    PyObject *c = NULL; // Note how I use PyObject *
    int c_int = 0; // Note how I have an accompanying int

    if (!PyArg_ParseTuple(args, "ss|O", &a, &b, &c) {
        return NULL;
    }

    // Ugly code starts here
    if (c) {
        if (c != Py_None) {
            if (!PyInt_Check(c)) {
                PyErr_SetString(PyExc_TypeError, "c must be int or None");
                return;
            }
            c_int = PyInt_AsSize_t(c);
        }
    }

    printf("c_int is %i
", c_int);

    //some_function_requiring_int_data_type(c_int);
}

And its use:

>>>from spam import foo
>>>foo('a', 'b')
c is 0
>>>foo('a', 'b', 100)
c is 100
>>>foo('a', 'b', None)
c is 0

DavidW · Accepted Answer

My first suggestion is to use keyword only arguments. The main advantage of this is to avoid ever having to pass in None placeholder values, since you never have to "fill in" (say) an unspecified 3rd positional argument just so you can specify the 4th. It's basically changing the Python interface to "match what you mean" a bit more.

static PyObject* int_from_kw(PyObject* self, PyObject* args, PyObject* kwargs) {
    char *a, *b;
    Py_ssize_t c = 0; // default value

    char* kwarg_names[] = {"a","b","c",NULL};

    // optional check to ensure c is passed only as a keyword argument - not needed with Python 3
    if (PyTuple_Size(args)>2) {
        PyErr_SetString(PyExc_TypeError,"Only two positional arguments allowed");
        return NULL;
    }

    if (!PyArg_ParseTupleAndKeywords(args,kwargs,"ss|i",kwarg_names,&a,&b,&c)) {
        return NULL;
    }
    printf("c_int is %li
", c);
    return PyLong_FromSsize_t(c);
}

(In Python 3 you can remove the length check and use "ss|$i" to specify that arguments after the $ are keyword only, which is a bit nicer). You need to specify the function type as METH_VARARGS|METH_KEYWORDS.

You can then call it from Python as

int_from_kw("something","something else") # default c
int_from_kw("something","something else",c=5)
int_from_kw(a="something",b="something else",c=5) # etc

but not

int_from_kw("something","something else",c="not an int")
int_from_kw("something","something else",5)

The downside is that the approach doesn't always work - sometimes you need the function to conform to a fixed interface that a 3rd party library enforces.

My second suggestion using a converter function. This doesn't eliminate any boiler plate, but keeps it all in one well contained and re-usable place. The version here is for Python 3 (because that's what I have installed!) but I think the main Python 2 change is to replace PyLong with PyInt.

int int_or_none(PyObject* o, void* i) {
    Py_ssize_t tmp;
    Py_ssize_t* i2 = i;
    if (o==Py_None) {
        return 1; // happy - leave integer as the default
    }
    if (PyLong_Check(o)) {
        tmp = PyLong_AsSize_t(o);
        if (PyErr_Occurred()) {
           return 0;
        } else {
           *i2 = tmp;
           return 1;
        }
    }
    PyErr_SetString(PyExc_TypeError, "c must be int or None");
    return 0; // conversion failed
}

static PyObject* test_int_none(PyObject* self, PyObject* args) {
    char *a, *b;
    Py_ssize_t c = 0; // default value

    if (!PyArg_ParseTuple(args, "ss|O&", &a, &b, int_or_none, &c)) {
        return NULL;
    }
    printf("c_int is %i
", c);
    return PyLong_FromSsize_t(c);
}

Some brief notes (with reference to your version):

We are confident that o is never NULL since it comes from Python which will always give you an object.
In the event of failure or None we don't change the pointer. This allows the default to be set in the calling function.
After conversion to the C integer type we have to check if an error occurred since you can get overflow errors if the integer is too large. In this case the correct exception is already set so we only have to return 0 to indicate failure. (I think this is less of a concern with Python 2 since it uses separate large and small integer types)

Neither of these suggestions really answers the question as asked, but they do provide what I think are cleaner alternatives.

How to permit None or a specific type as an argument to a Python C Extension function?

Answers (2)

Related Questions