CODEWITHSUNDEEP
pythonpython-3.xgeneratorretry-logicpython-tenacity
Bibimrlt
Bibimrlt

Reputation: 190

"retry" decorator from Tenacity doesn't work with a generator

It seems the @retry decorator does not seem to work when I use a generator.

I have a code sample to illustrate my situation:

from tenacity import retry, wait_exponential

@retry(wait=wait_exponential(multiplier=1, min=1, max=1))
def test_retry():
    print("test retry from tenacity")
    for i in range(10):
        if i == 0: raise Exception
        yield i

def gen():
    yield from test_retry()

bar = gen()
for foo in bar:
    print(foo)

When it raises the Exception, it doesn't retry.

Does somebody have an idea why this is not working?

Upvotes: 4

Views: 2439

Answers (1)

ingyhere
ingyhere

Reputation: 13831

It's a bug/feature/quagmire with Tenacity itself where retry logic fails on generator functions. Tenacity devs state that this is because "generators use exceptions internally." The original developer further goes on the write that "tenacity.retry() wrapped the generator function, not the generator itself (i.e. the user's code)." Basically there doesn't look to be any plans to work around this behavior, if it can even be done.

To deal with it, Tenacity annotations should be added to methods calling the generators -- where Exceptions can be caught readily by Tenacity as they bubble up through the call stack. Important: The generator functions must not hide exceptions, either.

# in generator do this and add retry annotations to calling method
...
try: 
    do_something()
except Exception as ex: 
    log_or_do_something_else()
    raise
finally: 
    cleanup()
yield something
...


# in generator don't do this
...
try: 
    do_something()
except Exception as ex: 
    pass
yield something
...

Upvotes: 3

Related Questions