Why does unpacking a chain with a non-iterable argument raise this error?

Question

1. Consider the following code:

   from itertools import chain
   list(chain(42))
   

   I am passing a non-iterable as an argument to chain and little surprisingly, I get exactly this error:

   TypeError: 'int' object is not iterable
   

   (Passing to list is only necessary because chain does not evaluate its arguments until the actual iteration.)

2. If I use chain correctly, I can unpack the result as function argument:

   from itertools import chain
   foo = lambda x: x
   foo(*chain([42]))
   

   This runs without errors.

3. Now, consider the combination of the two above cases, i.e., a chain with a non-iterable argument unpacked as function arguments:

   from itertools import chain
   foo = lambda x: x
   foo(*chain(42))
   

   As expected, this fails. In Python 3 this throws the same error as the first case. However, in Python 2.7.12, the error thrown is:

   TypeError: <lambda>() argument after * must be an iterable, not itertools.chain
   

   This does not make any sense to me. itertools.chain clearly is an iterable type: isinstance(chain(42),collections.Iterable) yields True. Also, it did not cause any problem in the second example. I would expect a similar error message as in case 2 or Python 3. What is the explanation for this error message?

Answer 1

The behaviour you are seeing is an attempt to give a clearer error message about what went wrong with the function call.

Python 2.7's way of determining if an object is iterable is just attempting to iterate it, and then catch the TypeError exception if necessary. It's not actually implemented in Python code, but that's still what happens in handling the function call syntax. Note: this has nothing to do with lambda, and a plain old def would have illustrated the example as well.

The function call is handled in CPython 2.7 by this C code:

static PyObject *
ext_do_call(PyObject *func, PyObject ***pp_stack, int flags, int na, int nk)
{
    ... snip ...

        t = PySequence_Tuple(stararg);
        if (t == NULL) {
            if (PyErr_ExceptionMatches(PyExc_TypeError) &&
                    /* Don't mask TypeError raised from a generator */
                    !PyGen_Check(stararg)) {
                PyErr_Format(PyExc_TypeError,
                             "%.200s%.200s argument after * "
                             "must be an iterable, not %200s",
                             PyEval_GetFuncName(func),
                             PyEval_GetFuncDesc(func),
                             stararg->ob_type->tp_name);
            }
            goto ext_call_fail;

    ... snip ...
}

I've truncated the code for brevity to show the relevant block: the starargs are iterated into a tuple, and if that fails with PyExc_TypeError then a new error is raised with the type and message matching what you've seen.

In Python 3, the function call C code was cleaned up and simplified significantly. Actually the ext_do_call function doesn't even exist any more, it was likely removed during implementation of PEP 3113. Now the exception from iterating a broken chain bubbles up unhandled. If you want to poke around in the current call code, you may start digging in Python/ceval.c::do_call_core.