CODEWITHSUNDEEP
pythonfunctionargumentspython-internalsoperator-precedence
mkrieger1
mkrieger1

Reputation: 23148

Evaluation order of positional and keyword arguments

Consider this contrived* example:

def count(one, two, three):
    print one
    print two
    print three

Three shall be the number thou shalt count, and the number of the counting shall be three.

>>> x = [1, 2, 3]
>>> count(*map(int, x), three=x.pop())
1
2
3

Four shalt thou not count,

>>> x = [1, 2, 3, 4]
>>> count(*map(int, x), three=x.pop())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: count() got multiple values for keyword argument 'three'

neither count thou two, excepting that thou then proceed to three.

>>> x = [1, 2]
>>> count(*map(int, x), three=x.pop())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: count() takes exactly 3 arguments (2 given)

Five is right out.

>>> x = [1, 2, 3, 4, 5]
>>> count(*map(int, x), three=x.pop())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: count() takes exactly 3 arguments (5 given)

After having read this question, I would actually have thought that x = [1, 2] is the only one that works, because

My expectation for x = [1, 2, 3] was to get the error that I actually saw for x = [1, 2, 3, 4].

What is going on here? Why are the arguments seemingly not evaluated from left to right? Are keyword arguments evaluated first?

*actually my real code corresponds to x = [1, 2, 3], which works, but I wasn't sure it was safe, and after reading the other question I thought it shouldn't actually work.

I'm using Python 2.7, if that matters.

Upvotes: 2

Views: 813

Answers (1)

Ashwini Chaudhary
Ashwini Chaudhary

Reputation: 250911

Python 2.7

If we look at the CPython source related to creating the AST(ast_for_call) for a function call the order of argument evaluation turns out to be:

return Call(func, args, keywords, vararg, kwarg, func->lineno,
                func->col_offset, c->c_arena);

ie. args --> keywords --> vararg --> kwarg

So, in your case the keyword argument is evaluated first and then the star based expression(vararg) is evaluated.

Byte code:

>>> dis.dis(lambda: func(1, 2, *('k', 'j', 'l'), z=1, y =2, three=x.pop(), **{kwarg:1}))
  1           0 LOAD_GLOBAL              0 (func)
              3 LOAD_CONST               1 (1)         # arg
              6 LOAD_CONST               2 (2)         # arg
              9 LOAD_CONST               3 ('z')       # keyword
             12 LOAD_CONST               1 (1)
             15 LOAD_CONST               4 ('y')       # keyword
             18 LOAD_CONST               2 (2)
             21 LOAD_CONST               5 ('three')   # keyword
             24 LOAD_GLOBAL              1 (x)
             27 LOAD_ATTR                2 (pop)
             30 CALL_FUNCTION            0
             33 LOAD_CONST               9 (('k', 'j', 'l')) #vararg
             36 BUILD_MAP                1
             39 LOAD_CONST               1 (1)
             42 LOAD_GLOBAL              3 (kwarg)     #kwarg
             45 STORE_MAP
             46 CALL_FUNCTION_V

Hence in your case the pop() call will happen first followed by the varargs evaluation.

So, if three is a part of kwargs then we will get an error with map:

>>> x = [1, 2, 3]
>>> count(*map(float, x), **{'three': x.pop()})
Traceback (most recent call last):
  File "<ipython-input-133-e8831565af13>", line 1, in <module>
    count(*map(float, x), **{'three': x.pop()})
TypeError: count() got multiple values for keyword argument 'three'

It will work if we do it *lazily:

>>> x = [1, 2, 3]
>>> count(*(float(y) for y in x), **{'three': x.pop()})
1.0, 2.0, 3

*The reason why generator works and map or list comprehension fails is explained at the end.

Python 3.5

The ast_for_call function here only maintains two lists: args and keywords.

Here the varargs are inserted into the args list and kwargs go to the keywords list. So, in the end the call looks like:

return Call(func, args, keywords, func->lineno, func->col_offset, c->c_arena);

Byte code:

>>> dis.dis(lambda: func(1, 2, *('k', 'j', 'l'), z=1, y =2, three=x.pop(), **{kwarg:1}))
  1           0 LOAD_GLOBAL              0 (func)
              3 LOAD_CONST               1 (1)
              6 LOAD_CONST               2 (2)
              9 LOAD_CONST               9 (('k', 'j', 'l'))
             12 LOAD_CONST               6 ('z')
             15 LOAD_CONST               1 (1)
             18 LOAD_CONST               7 ('y')
             21 LOAD_CONST               2 (2)
             24 LOAD_CONST               8 ('three')
             27 LOAD_GLOBAL              1 (x)
             30 LOAD_ATTR                2 (pop)
             33 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             36 LOAD_GLOBAL              3 (kwarg)
             39 LOAD_CONST               1 (1)
             42 BUILD_MAP                1
             45 CALL_FUNCTION_VAR_KW   770 (2 positional, 3 keyword pair)
             48 RETURN_VALUE

Now things can get a little exciting if the expression yielding the varargs is lazy:

>> def count(one, two, three):
        print (one, two, three)
...
>>> x = [1, 2, 3]
>>> count(*map(float, x), three=x.pop())  # map is lazy in Python 3
1.0 2.0 3
>>> x = [1, 2, 3]
>>> count(*[float(y) for y in x], three=x.pop())
Traceback (most recent call last):
  File "<ipython-input-25-b7ef8034ef4e>", line 1, in <module>
    count(*[float(y) for y in x], three=x.pop())
TypeError: count() got multiple values for argument 'three'

Byte code:

>>> dis.dis(lambda: count(*map(float, x), three=x.pop()))
  1           0 LOAD_GLOBAL              0 (count)
              3 LOAD_GLOBAL              1 (map)
              6 LOAD_GLOBAL              2 (float)
              9 LOAD_GLOBAL              3 (x)
             12 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
             15 LOAD_CONST               1 ('three')
             18 LOAD_GLOBAL              3 (x)
             21 LOAD_ATTR                4 (pop)
             24 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             27 CALL_FUNCTION_VAR      256 (0 positional, 1 keyword pair)
             30 RETURN_VALUE
>>> dis.dis(lambda: count(*[float(y) for y in x], three=x.pop()))
  1           0 LOAD_GLOBAL              0 (count)
              3 LOAD_CONST               1 (<code object <listcomp> at 0x103b63930, file "<ipython-input-28-1cc782164f20>", line 1>)
              6 LOAD_CONST               2 ('<lambda>.<locals>.<listcomp>')
              9 MAKE_FUNCTION            0
             12 LOAD_GLOBAL              1 (x)
             15 GET_ITER
             16 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             19 LOAD_CONST               3 ('three')
             22 LOAD_GLOBAL              1 (x)
             25 LOAD_ATTR                2 (pop)
             28 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             31 CALL_FUNCTION_VAR      256 (0 positional, 1 keyword pair)
             34 RETURN_VALUE

The lazy call works because unpacking(aka actual evaluation of the generator) doesn't happen until the function is actually called, hence in this case pop() call will remove the 3 first and then later on map will only pass 1, 2.

But, in the case of list comprehension the list object already contains 3 items and then even though pop() removed 3 later on we are still passing two values for the third argument.

Upvotes: 2

Related Questions