CODEWITHSUNDEEP
pythondjangopython-3.xgunicornmultiple-processes
BAE
BAE

Reputation: 8936

worker_exit of Gunicorn

I am using Gunicorn to deploy an Django app.

There is a global singleton variable defined in util.py:

global SNS_PRODUCERS
SNS_PRODUCERS = {}
def close_producers():
    logger.info('Closing all producers...')
    global SNS_PRODUCERS
    for name, producer in SNS_PRODUCERS.items():
        producer.close()

I would like to call close_producers when my app shutdown. I added the following into gunicornconf.py:

def child_exit(server, worker):
    # Join background threads in util.SNS_PRODUCERS
    try:
        from util import close_producers
        close_producers()
    except:
        pass


def worker_exit(server, worker):
    try:
        from util import close_producers
        close_producers()
    except:
        pass

My understanding is that, there is one master process, and multiple worker process forked from master process. In each process, util.py is imported to this process. So, there are multiple SNS_PRODUCERS in these process, one SNS_PRODUCERS in one worker or master process.

Each worker process has ITS OWN worker_exit hook. The master process has ITS OWN child_exit hook.

When one worker is shutdown, its worker_exit is called, and then ITS OWN SNS_PRODUCERS is closed. Other SNS_PRODUCERS are still open.

Am I right? Any corrections are welcomed.

Upvotes: 1

Views: 1810

Answers (1)

kdavh
kdavh

Reputation: 414

Just one correction / clarification: The master process isn't running any application code, it's a simple process that manages signal handling.

The child_exit hook is run in the master process, so won't have access to any memory or global variables like SNS_PRODUCERS from any child / worker processes. The only hook necessary here is worker_exit which correctly calls your close_producers method as you illustrated.

Upvotes: 0

Related Questions