CODEWITHSUNDEEP
pythonnginxflaskuwsgi
fabbronet
fabbronet

Reputation: 21

nginx + uwsgi + flask and multiprocess

I have I python-flask script running under uwsgi + nginx deployment configuration. My uwsgi.ini file:

[uwsgi]
pythonpath=/usr/bin/python3
socket=/tmp/grace.sock
chmod-socket = 666
vacuum = true
uid = www-data 
gid = www-data
plugin= python3
chdir= /home/grace/pyRep/beta_grace
module= app:app
enable-threads= true
master= true
processes= 3
#cheaper= 1
logto = /home/grace/pyRep/beta_grace/uwsgi.log
lazy-apps = true
single-interpreter=true

now, inside my script I have a function like this:

from uwsgidecorators import *

@timer(60)    
def foo():
    global_var += 1
    print(global_var)

Looking at my logs i find: global_var: 1 global_var: 1 global_var: 1

In my opinion this is due to the lazy-apps options enabled, so after fork i have three copies of this task running and after some time I find:

global_var: 34 global_var: 32 global_var: 32

I tried with the @lock and @postfork decorator before @timer decorator but nothing changes. If I take out lazy-apps option I have problems connecting to the mongoDB engine and other weird behaviours so I think that I have to keep it. The only solution I found is to limit processes to 1 but this obiouvsly decreases performances. Any advice?!

Upvotes: 1

Views: 1742

Answers (2)

fabbronet
fabbronet

Reputation: 21

I think I found a less invasive solution using a single process and a mule so now my uwsg.ini:

processes=1
mules=1

and my python script:

@timer(60,target='mule')

This way I offloaded my main process binding the timer to the mule and other tasks on the main process. I thought of using 2 processes+1 mule but
also with only one process the speed is ok!

Upvotes: 1

Larry.He
Larry.He

Reputation: 634

The key point is that you can not share/use simple variable(even global) to communicate between multiple processes.

IPC methods can be found here

For your case, I think redis is one solution, remember to use distributed lock

Upvotes: 0

Related Questions