CODEWITHSUNDEEP
pythonlinuxmultiprocessingkillpython-requests
neutrinus
neutrinus

Reputation: 2009

Strange problems when using requests and multiprocessing

Please check this python code:

#!/usr/bin/env python
import requests
import multiprocessing
from time import sleep, time
from requests import async

def do_req():
    r = requests.get("http://w3c.org/")

def do_sth():    
    while True:
        sleep(10)

if __name__ == '__main__':        
    do_req()
    multiprocessing.Process( target=do_sth, args=() ).start()

When I press Ctrl-C (wait 2sec after run - let Process run), it doesn't stop. When I change the import order to:

from requests import async
from time import sleep, time

it stops after Ctrl-C. Why it doesn't stop/kill in first example?

It's a bug or a feature?

Notes:

Upvotes: 4

Views: 1314

Answers (1)

Piotr Duda
Piotr Duda

Reputation: 1815

Requests async module uses gevent. If you look at the source code of gevent you will see that it monkey patches many of Python's standard library functions, including sleep:

request.async module during import executes:

    from gevent import monkey as curious_george
    # Monkey-patch.
    curious_george.patch_all(thread=False, select=False)

Looking at the monkey.py module of gevent you can see:

https://bitbucket.org/denis/gevent/src/f838056c793d/gevent/monkey.py#cl-128

def patch_time():
    """Replace :func:`time.sleep` with :func:`gevent.sleep`."""
    from gevent.hub import sleep
    import time
    patch_item(time, 'sleep', sleep)

Take a look at the code from the gevent's repository for details.

Upvotes: 6

Related Questions