CODEWITHSUNDEEP
pythonexceptionfabricslack
Loknar
Loknar

Reputation: 1189

Python fabric, send notification on failure

I've been trying to figure out what the best way is to do something when my fabric script fails (for example send a slack notification message via python module slackbot).

I've made an example where I try to do the above here:

fab_failtest.py my_slackclient.py

You can run above example by downloading both files to a directory, pip install fabric and slackbot, then run:

fab --fabfile=fab_failtest.py fail_test1 or

fab --fabfile=fab_failtest.py fail_test2

(you also have to have a machine you can ssh to, in this example I have mrbluesky@elo with open ssh port on 22)

At first I thought I had it with the fail_test1 example but I've seen it fail several times to send the slack message on failure, I'm wondering if there might be a race condition or something involved? I could start using fail_test2 instead but I really like to have access to the stack-trace like in fail_test1.

Is there a better way to do this, like, something provided in python fabric that does excatly what I'm trying to accomplish in above example?

Upvotes: 3

Views: 2252

Answers (1)

Javier Buzzi
Javier Buzzi

Reputation: 6808

I disagree with both your approaches. Im a strong believer that less code is better. What do i mean by that? A function should do what its name says, no more no less, if you have to add in a global handler like that i would add it in as a wrapper, fabric functions are difficult enough to read, no need to add error handling to the mix. With that said:

import sys
import traceback

from fabric.api import task, settings, local, abort
from fabric.decorators import _wrap_as_new
from functools import wraps

HOST = 'elo'
PORT = 22


def alert_on_fail(func):
    @wraps(func)
    def decorated(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        except:
            # TODO: add more code here
            exception_type, value, tb_msg = sys.exc_info()
            traceback_msg = traceback.format_exc()
            notify('something went wrong: ' + traceback_msg)
            abort('exiting error!!')
    return _wrap_as_new(func, decorated)


@task
@alert_on_fail
def fail_test(host=HOST, port=PORT):
    notify('fail test', msg_type='info')
    local('''python -c "raise Exception('foobar')"''')
    notify('script ran successfully', msg_type='success')  # this will never run because the function above crashed


@task
@alert_on_fail
def pass_test(host=HOST, port=PORT):
    notify('pass test', msg_type='info')
    local('whoami')
    notify('script ran successfully', msg_type='success')


def notify(msg, **kwargs):
    # DISREGARD THIS
    print 'sent to slack:', msg

Output:

$ fab fail_test
sent to slack: fail test
[localhost] local: python -c "raise Exception('foobar')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
Exception: foobar

Fatal error: local() encountered an error (return code 1) while executing 'python -c "raise Exception('foobar')"'

Aborting.
sent to slack: something went wrong: Traceback (most recent call last):
  File "/private/tmp/fabfile.py", line 21, in decorated
    return func(*args, **kwargs)
  File "/private/tmp/fabfile.py", line 34, in fail_test
    local('''python -c "raise Exception('foobar')"''')
  File "/usr/local/lib/python2.7/site-packages/fabric/operations.py", line 1198, in local
    error(message=msg, stdout=out, stderr=err)
  File "/usr/local/lib/python2.7/site-packages/fabric/utils.py", line 347, in error
    return func(message)
  File "/usr/local/lib/python2.7/site-packages/fabric/utils.py", line 53, in abort
    sys.exit(msg)
SystemExit: local() encountered an error (return code 1) while executing 'python -c "raise Exception('foobar')"'


Fatal error: exiting error!!

Aborting.
exiting error!!

and:

$ fab pass_test
sent to slack: pass test
[localhost] local: whoami
buzzi
sent to slack: script ran successfully

Done.

You'll notice that the functions are now "easy" to read, they are "simple", all the error handling code has been moved over somewhere else.

Upvotes: 2

Related Questions