CODEWITHSUNDEEP
pythonmultithreadinguwsgi
J Aamish
J Aamish

Reputation: 532

Python uwsgi threads in workers

New to uwsgi and when I run the below code 

from time import sleep
import threading
import os
import sys


i = 0


def daemon():
    print("pid ", os.getpid())
    global i
    while True:
        i += 1
        print("pid ", os.getpid(), i)
        sleep(3)


th = threading.Thread(target=daemon, args=())
th.start()

print("application pid ", os.getpid())


def application(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/html')])
    return [str(i).encode()]


def atexit(*args):
    import uwsgi
    print('At exit called ', uwsgi.worker_id(), os.getpid())
    sys.exit(1)


try:
    import uwsgi
    uwsgi.atexit = atexit
except ImportError:
    pass

using the command 

uwsgi --http :9090 --wsgi-file  wsgi.py  --enable-threads --processes 2

I get the output

*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 1418
your memory page size is 4096 bytes
detected max file descriptor number: 256
lock engine: OSX spinlocks
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :9090 fd 4
spawned uWSGI http 1 (pid: 37671)
uwsgi socket 0 bound to TCP address 127.0.0.1:62946 (port auto-assigned) fd 3
Python version: 3.7.4 
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145776 bytes (142 KB) for 2 cores
*** Operational MODE: preforking ***
pid  37670
application pid  37670
pid  37670 1
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x7fd29af02010 pid: 37670 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (pid: 37670, cores: 1)
spawned uWSGI worker 2 (pid: 37672, cores: 1)

pid  37670 2
pid  37670 3
pid  37670 4

I get two worker processes (37670 and 37672). But the thread that I’m creating th = threading.Thread(target=daemon, args=()) is running only in the first worker process (37670) but not in 37672...

Question:

  1. I was expecting another thread also to be running in the 2nd worker process, but obviously that is not the case. How is it working internally?
  2. How could I create threads for each worker process and master process separately (in this example, I’m not running a master process)

Upvotes: 5

Views: 7455

Answers (1)

timedacorn
timedacorn

Reputation: 548

This seems to be because of preforking.uwsgi works by loading the application once ( hence only one interpreter) and then forking depending on the processes needed. If we use lazy-apps,

uwsgi --http :9090 --wsgi-file  wsgi.py  --enable-threads --processes 2 --lazy-apps

this loads one application per process and the thread runs in both worker processes.More info here.

Output

*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (pid: 14983, cores: 1)
spawned uWSGI worker 2 (pid: 14985, cores: 1)
pid 14983
pid 14985
Application pid 14983
Application pid 14985
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x55e8d5282bb0 pid: 14985 (default app)
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x55e8d5282bb0 pid: 14983 (default app)
pid 14985 1
pid 14983 1
pid 14985 2
pid 14983 2
pid 14985 3
pid 14983 3

You can create multiple threads per process by using the --threads option in uwsgi

--processes 2 --threads 2

this creates 2 threads for each process.

Upvotes: 6

Related Questions