More pythonic way to handle nested try... except blocks?

Question

Is there a cleaner or more pythonic way to do the following?

try:
    error_prone_function(arg1)
except MyError:
    try:
        error_prone_function(arg2)
    except MyError:
        try:
            another_error_prone_function(arg3)
        except MyError:
            try:
                last_error_prone_function(arg4)
            except MyError:
                raise MyError("All backup parameters failed.")

Basically it's: If attempt #1 fails, try #2. If #2 fails, try #3. If #3 fails, try #4. If #4 fails, ... if #n fails, then finally raise some exception.

Note that I'm not necessarily calling the same function every time, nor am I using the same function arguments every time. I am, however, expecting the same exception MyError on each function.

Answer 1

A generator based approach might give you a little more flexibility than the data-driven approach:

def attempts_generator():

#   try:
#       <the code you're attempting to run>
#   except Exception as e:
#       # failure
#       yield e.message
#   else:
#       # success
#       return

    try:
        print 'Attempt 1'
        raise Exception('Failed attempt 1')
    except Exception as e:
        yield e.message
    else:
        return

    try:
        print 'Attempt 2'
        # raise Exception('Failed attempt 2')
    except Exception as e:
        yield e.message
    else:
        return

    try:
        print 'Attempt 3'
        raise Exception('Failed attempt 3')
    except Exception as e:
        yield e.message
    else:
        return

    try:
        print 'Attempt 4'
        raise Exception('Failed attempt 4')
    except Exception as e:
        yield e.message
    else:
        return

    raise Exception('All attempts failed!')

attempts = attempts_generator()
for attempt in attempts:
    print attempt + ', retrying...'

print 'All good!'

The idea is to build a generator that steps through attempt blocks via a retry loop.

Once the generator hits a successful attempt it stops its own iteration with a hard return. Unsuccessful attempts yield to the retry loop for the next fallback. Otherwise if it runs out of attempts it eventually throws an error that it couldn't recover.

The advantage here is that the contents of the try..excepts can be whatever you want, not just individual function calls if that's especially awkward for whatever reason. The generator function can also be defined within a closure.

As I did here, the yield can pass back information for logging as well.

Output of above, btw, noting that I let attempt 2 succeed as written:

mbp:scratch geo$ python ./fallback.py
Attempt 1
Failed attempt 1, retrying...
Attempt 2
All good!

If you uncomment the raise in attempt 2 so they all fail you get:

mbp:scratch geo$ python ./fallback.py
Attempt 1
Failed attempt 1, retrying...
Attempt 2
Failed attempt 2, retrying...
Attempt 3
Failed attempt 3, retrying...
Attempt 4
Failed attempt 4, retrying...
Traceback (most recent call last):
  File "./fallback.py", line 47, in <module>
    for attempt in attempts:
  File "./fallback.py", line 44, in attempts_generator
raise Exception('All attempts failed!')
Exception: All attempts failed!

Edit:

In terms of your pseudocode, this looks like:

def attempts_generator():
    try:
        error_prone_function(arg1)
    except MyError
        yield
    else:
        return

    try:
        error_prone_function(arg2)
    except MyError
        yield
    else:
        return

    try:
        another_error_prone_function(arg3)
    except MyError:
        yield
    else:
        return

    try:
        last_error_prone_function(arg4)
    except MyError:
        yield
    else:
        return

    raise MyError("All backup parameters failed.")

attempts = attempts_generator()
for attempt in attempts:
    pass

It'll let any exception but MyError bubble out and stop the whole thing. You also could choose to catch different errors for each block.

Answer 2

Thanks to John Kugelman's post here, I decided to go with this which utilizes the lesser-known else clause of a for loop to execute code when an entire list has been exhausted without a break happening.

funcs_and_args = [(func1, "150mm"),
                  (func1, "100mm",
                  (func2, "50mm"),
                  (func3, "50mm"),
                  ]
for func, arg in funcs_and_args :
    try:
        func(arg)
        # exit the loop on success
        break
    except MyError:
        # repeat the loop on failure
        continue
else:
    # List exhausted without break, so there must have always been an Error
    raise MyError("Error text")

As Daniel Roseman commented below, be careful with indentation since the try statement also has an else clause.