CODEWITHSUNDEEP
pythonperformancedecorator
dochead
dochead

Reputation: 1825

How much overhead do decorators add to Python function calls

I've been playing around with a timing decorator for my pylons app to provide on the fly timing info for specific functions. I've done this by creating a decorator & simply attaching it to any function in the controller I want timed.

It's been pointed out however that decorators could add a fair amount of overhead to the call, and that they run 2-3x slower than an undecorated function.

Firstly, I would expect that executing a decorated function would take a smite longer than an undecorated one, but I would expect that overhead to be in the thousandths of seconds & be negligible compared to a SQL insert call. The decorator itself does simple simple timing calculations using time.time() & some very simple aggregation.

Do decorators add significant overhead to a system? I can't find anything to back that up.

Upvotes: 23

Views: 7723

Answers (4)

Kuchara
Kuchara

Reputation: 715

Out of curiosity I've just tried to experiment on this, and:

import timeit

def my_function_undecorated():
    pass
    
def timing_decorator(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

@timing_decorator
def my_function():
    pass

reps = 10**6

decorated_time = timeit.timeit("my_function()", globals=globals(), number=reps) / reps
undecorated_time = timeit.timeit("my_function_undecorated()", globals=globals(), number=reps) / reps

print(f"Decorated function took {decorated_time} seconds to execute")
print(f"Undecorated function took {undecorated_time} seconds to execute")
print(f"Absolute diff: {decorated_time - undecorated_time} seconds")
print(f"Relative diff: {decorated_time/undecorated_time * 100} %")

And results (Python 3.10.6 @ WSL2 @ Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz):

Decorated function took 1.359141360007925e-07 seconds to execute
Undecorated function took 5.3043514999444594e-08 seconds to execute
Absolute diff: 8.287062100134791e-08 seconds
Relative diff: 256.23139040128774 %

So decorator's overhead is ~8.122772299975623e-08 seconds, which is more than 250% of the undecorated variant for a function that does nothing, and a wrapper that just returns.

Of course as others say, it depends on what wrapper or original function does. This can be anything:

  • much more than 2x (e.g. heavy computations or I/O inside wrapper)
  • much fewer than 1x (e.g. see @functools.cache)

Upvotes: 1

Eric O. Lebigot
Eric O. Lebigot

Reputation: 94485

What is important to know is that a decorator has a simple effect:

@decorator
def f():
    …

is just syntactic sugar for

def f():
    …
f = decorator(f)

Thus, if decorator does not do anything, you do not have any overhead when calling the decorated function (the call decorator(f) does take a little time, though), like in

decorator = lambda func: func
@decorator
def f():
    …

If the decorator does anything, you only get whatever time overhead the decorator involves. This typically include an additional function call (that of the decorated function), as in

def decorator(func):
    def decorated_func():
        print "Before calling function", func  # Some overhead (but that's normal)
        func()  # This will be a second function call, after the call to decorated_func()
    return decorated_func

Thus, in itself, decorating a function does not add much overhead for what you want to do: the only obvious overhead that you could in principle remove would be to not call func() in the decorated function but instead copy its full code, but the legibility of the code would suffer (legibility and flexibility are some of the reasons decorators do exist in the first place).

Upvotes: 4

S.Lott
S.Lott

Reputation: 391854

Do decorators add significant overhead to a system? I can't find anything to back that up.

They add almost no measurable overhead. Zero.

It's important to note that the decorator runs once to create the decorated function. Once.

The decorated function has two parts to it.

  1. whatever decoration was added. This is not overhead.

  2. plus the original function. This is not overhead.

There's no real overhead at all. You might -- with some care -- be able to measure the overhead of one extra function call-and-return as part of a decorated function, but that's almost unmeasurably small. And it's probably far less than an alternative design that doesn't use decoration.

Upvotes: 1

John La Rooy
John La Rooy

Reputation: 304175

The overhead added by using a decorator should be just one extra function call.

The work being done by the decorator isn't part of the overhead as your alternative is to add the equivalent code to the decorated object.

So it's possible that the decorate function takes twice as long to run, but that's because the decorator is doing some important work that takes roughly the same time to fun as the undecorated function.

Upvotes: 16

Related Questions