CODEWITHSUNDEEP
pythonperformancearguments
Grismar
Grismar

Reputation: 31354

Performance difference between positional and keyword arguments in Python?

There are many explanations of the difference between positional and keyword arguments for Python functions on StackOverflow and the difference in utility is clear to me. However, I was unable to find a clear answer (and perhaps explanation of) whether positional arguments are substantially faster that keyword arguments in some situations.

On the one hand, I can imagine they would be, as there has to be no work done to look up the arguments passed. But on the other hand, unless the arguments are passed as a dict spread into the **kwargs, or something like that, I imagine the compiler may just optimise this problem away and the bytecode might perform nearly identical for base cases.

So:

from timeit import timeit


def fun1(a, b, c):
    pass  # or some operation that wouldn't get this function optimised out altogether


def fun2(a, /, b, c=3):
    pass  # or some operation that wouldn't get this function optimised out altogether


fun1(1, 2, 3)  # call 1
fun2(1, 2, 3)  # call 2
fun2(1, 2, c=3)  # call 3
fun2(1, b=2, c=3)  # call 4
d = {'c': 3}
fun2(1, 2, *d)  # call 5

My question is: which ones of these calls are slower than the others and why? And am I missing a key example that would focus on the worst performance hit here?

I would assume call 5 is clearly worst, as Python needs to assess at runtime what parameters have been passed. But it depends on optimisation whether calls 4 and 3 are dealt with more efficiently. And for call 1 to be faster than call 2, there would have to be little or no optimisation at all.

I appreciate that the answer may depend on the specific Python compiler / VM implementation, so I'm asking in particular about the default Python interpreter that most people will be using.

Of course one could just write a script to measure the difference, but I feel that's going to get muddied real quick by whatever the function does and perhaps some local circumstances, so I was hoping someone would read this question and actually know why there would be a difference once way or the other.

I did run a timer test and it confirmed my suspicions:

print(timeit(lambda: fun1(1, 2, 3), number=int(1e8)))  # call 1
print(timeit(lambda: fun2(1, 2, 3), number=int(1e8)))  # call 2
print(timeit(lambda: fun2(1, 2, c=3), number=int(1e8)))  # call 3
print(timeit(lambda: fun2(1, b=2, c=3), number=int(1e8)))  # call 4
d = {'c': 3}
print(timeit(lambda: fun2(1, 2, *d), number=int(1e8)))  # call 5

7.7741496
8.472946400000001
9.2335016
9.755567000000003
18.6909664

(Note that I did additional runs randomising the order in which these functions are called and the relative results remain more or less the same)

My own reasoning: since Python would have to do a lot of analysis to know how a function is called throughout the program being run, and some calls may only execute conditionally, it can't really optimise for how the function is called. So, keyword arguments always incur a penalty and using keyword arguments with their keyword adds a penalty.

As pointed out in the comments: if you care enough about performance for this to really matter, you may want to reconsider picking Python on the standard VM as a platform. And obsessing about this type of performance issue tends to be premature regardless. The question however is not so much "how do I optimise", or "should I optimise", but "why is there actually a performance difference"?

Thanks for any comments or explanations in an answer.

Upvotes: 7

Views: 945

Answers (1)

Minion Wayde
Minion Wayde

Reputation: 51

from dis import dis
def add(a,b): return a + b

call it with postional arguments

dis("add(1,2)")
  1           0 LOAD_NAME                0 (add)
              2 LOAD_CONST               0 (1)
              4 LOAD_CONST               1 (2)
              6 CALL_FUNCTION            2
              8 RETURN_VALUE

call it with keyword arguments

dis("add(1, b=2)")
  1           0 LOAD_NAME                0 (add)
              2 LOAD_CONST               0 (1)
              4 LOAD_CONST               1 (2)
              6 LOAD_CONST               2 (('b',))
              8 CALL_FUNCTION_KW         2
             10 RETURN_VALUE

As you can see here how bytecode structions show, if you call a function with keyword arguments, pvm has to do extra work.

what CALL_FUNCTION_KW does:

  1. Pops the callable object (the function to be called) off the stack.
  2. Pops the keyword arguments off the stack.
  3. Pops the positional arguments off the stack.
  4. Calls the function with the given positional and keyword arguments.
  5. Pushes the return value of the function onto the stack.

The creation of these data structure brings extra overhead

and in the case of

d = {'c': 3}
print(timeit(lambda: fun2(1, 2, *d), number=int(1e8)))  # call 5

*d would call d.__iter__ and d.__getitem__ so these overhead adds up.

Upvotes: 1

Related Questions