Apache, importing a python module with mutex and ladon, script hang

Question

i have a strange behaviour of a python script running through Apache/mod_wsgi/Ladon, when running it from a python shell, every thing is fine, on one machine everything is fine, but on the other the same script hang and it seems that it is indefinitely doing or waiting something: that is opening the page from 192.168.xxx.xxx/ws/ never end (browser are loading again and again, turning round, ie saying ). I can not find what happen, logging thing to the apache log show me nothing as the script imported seems to never end when used through apache/ladon. (and as both machines are configured the same way, i have plain headache with that :/)

as the script load some data from the filesystem and database, can the hanging be from locking these ressources indefinitely?

can i wrap the module that hang inside something like the lines below to be sure there is nothing locking? actually i have tried to do so, and it change nothing at all, but and this is my first question, is it right to write it like this (as i am a bit confued but different mutex processes)?

[...]
from multiprocessing import Lock
mutex = Lock()

mutex.acquire()
import MyHangingModule
mutex.release()
[...]

is there any case where when serving a page from apache may end with an infinite time out? that is the python mod_wsgi/ladon script is killed by apache or ladon while something is still waiting for a reply from the killed? as when shutting down the server i get some:

[Tue Jul 09 16:07:47 2013] [warn] child process 1676 still did not exit, sending a SIGTERM
[Tue Jul 09 16:07:49 2013] [warn] child process 1676 still did not exit, sending a SIGTERM
[Tue Jul 09 16:07:51 2013] [warn] child process 1676 still did not exit, sending a SIGTERM
[Tue Jul 09 16:07:53 2013] [error] child process 1676 still did not exit, sending a SIGKILL
[Tue Jul 09 16:07:54 2013] [notice] caught SIGTERM, shutting down

(by searching about pid, through mod_status, it is actually the handing script staying in W mode)

EDIT: about wsgi, i have nothing Inside the wsgi.conf, everything is commented # actually, I install it from the default Debian installation. Is there something that i should look? or that I obviously miss?

for the other, it is also default Debian (actually no change inside config file, but specifically install apache2-mpm-worker):

# worker MPM
<IfModule mpm_worker_module>
    StartServers          2
    MinSpareThreads      25
    MaxSpareThreads      75 
    ThreadLimit          64
    ThreadsPerChild      25
    MaxClients          150
    MaxRequestsPerChild   0
</IfModule>

Output of /usr/sbin/apache2ctl -l

Compiled in modules:
  core.c
  mod_log_config.c
  mod_logio.c
  worker.c
  http_core.c
  mod_so.c

about mod for wsgi (without adding WSGIApplicationGroup %{GLOBAL}) to wsgi.conf):

<pre>
mod_wsgi.process_group = ['']
mod_wsgi.application_group = ['myserver|/mycfg'] # mycfg is actually the name of the python script while i have replaced the server name for posting here
wsgi.multithread = [True]
</pre>

about mod for wsgi (after adding WSGIApplicationGroup %{GLOBAL}) to wsgi.conf) (thus very slow for my others scripts):

<pre>
mod_wsgi.process_group = ['']
mod_wsgi.application_group = ['']
wsgi.multithread = [True]
</pre>

well, if it could help, here is the log part for a sessing using a webservice that may be slow : (actually i would have liked that the externalDataLoader.py loaded: 0:00:00.394606 (21941 entries loaded) was shared by every one and then occurs only one. Also i have deactivate all others python handled... can this be that the twomod_wsgi (pid=xxxxx): Attach interpreter ''`can then be using some ressources and waiting indefinetly without any response and thus making the whole server and others WS query to end with the "child process not exit" ?

[Wed Jul 24 10:49:20 2013] [notice] caught SIGTERM, shutting down
[Wed Jul 24 10:49:21 2013] [info] mod_wsgi (pid=11199): Initializing Python.
[Wed Jul 24 10:49:21 2013] [notice] Apache/2.2.16 (Debian) mod_wsgi/3.3 Python/2.6.6 configured -- resuming normal operations
[Wed Jul 24 10:49:21 2013] [info] Server built: Mar  3 2013 11:36:06
[Wed Jul 24 10:49:21 2013] [info] mod_wsgi (pid=11203): Initializing Python.
[Wed Jul 24 10:49:21 2013] [info] mod_wsgi (pid=11199): Attach interpreter ''.
[Wed Jul 24 10:49:21 2013] [info] mod_wsgi (pid=11203): Attach interpreter ''.
[Wed Jul 24 10:49:39 2013] [info] [client 192.168.1.130] mod_wsgi (pid=11199, process='', application=''): Loading WSGI script '/var/www/mywebserver/ws/wshandler.py'., referer: http://192.168.1.5/demo/search.html
[Wed Jul 24 10:49:40 2013] [error] externalDataLoader.py loaded: 0:00:00.394606 (21941 entries loaded)
[Wed Jul 24 10:49:42 2013] [error] WSforSearchInsideExternalData. loaded: 0:00:00.000032.
[Wed Jul 24 10:49:42 2013] [error] searched [test] in [External1]: 0:00:00.026233
[Wed Jul 24 10:49:42 2013] [error] searched [test] in [External2]: 0:00:00.067869
[Wed Jul 24 10:49:43 2013] [error] searched [test] in [External3]: 0:00:00.104222
[Wed Jul 24 10:49:43 2013] [error] searching done for [test]: 0:00:00.104222
[Wed Jul 24 10:49:59 2013] [info] [client 192.168.1.130] mod_wsgi (pid=11203, process='', application=''): Loading WSGI script '/var/www/mywebserver/ws/wshandler.py'., referer: http://192.168.1.5/demo/search.html
[Wed Jul 24 10:50:00 2013] [error] externalDataLoader.py loaded: 0:00:00.384349 (21941 entries loaded)
[Wed Jul 24 10:50:02 2013] [error] WSforSearchInsideExternalData. loaded: 0:00:00.000033.
[Wed Jul 24 10:50:02 2013] [error] searched [abjg] in [External1]: 0:00:00.026000
[Wed Jul 24 10:50:02 2013] [error] searched [abjg] in [External2]: 0:00:00.066215
[Wed Jul 24 10:50:02 2013] [error] searched [abjg] in [External3]: 0:00:00.100293
[Wed Jul 24 10:50:02 2013] [error] searching done for [abjg]: 0:00:00.100293
[Wed Jul 24 10:50:40 2013] [error] searched [abjg] in [External1]: 0:00:00.001513
[Wed Jul 24 10:50:40 2013] [error] searched [abjg] in [External2]: 0:00:00.080266
[Wed Jul 24 10:50:40 2013] [error] searched [abjg] in [External3]: 0:00:00.119280
[Wed Jul 24 10:50:40 2013] [error] searching done for [abjg]: 0:00:00.119280
[Wed Jul 24 10:50:48 2013] [error] searched [abje] in [External1]: 0:00:00.001491
[Wed Jul 24 10:50:48 2013] [error] searched [abje] in [External2]: 0:00:00.079586
[Wed Jul 24 10:50:48 2013] [error] searched [abje] in [External3]: 0:00:00.123212
[Wed Jul 24 10:50:48 2013] [error] searching done for [abje]: 0:00:00.123212

Answer 1

See and try the solution described in:

• http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API

You are likely using an extension module which will not work in a sub interpreter.

In short, add to the Apache configuration:

WSGIApplicationGroup %{GLOBAL}