CODEWITHSUNDEEP
pythonmultithreadingasynchronousgeventeel
Clay Records
Clay Records

Reputation: 503

Gevent functions aren't running asynchronously

In this Gevent tutorial it goes over how to spawn multiple threads to run asynchronously. 

import gevent
import random

def task(pid):
    """
    Some non-deterministic task
    """
    gevent.sleep(random.randint(0,2)*0.001)
    print('Task %s done' % pid)

def synchronous():
    for i in range(1,10):
        task(i)

def asynchronous():
    threads = [gevent.spawn(task, i) for i in xrange(10)]
    gevent.joinall(threads)

print('Synchronous:')
synchronous()

print('Asynchronous:')
asynchronous()

Results

Synchronous:
Task 1 done
Task 2 done
Task 3 done
Task 4 done
Task 5 done
Task 6 done
Task 7 done
Task 8 done
Task 9 done
Asynchronous:
Task 1 done
Task 5 done
Task 6 done
Task 2 done
Task 4 done
Task 7 done
Task 8 done
Task 9 done
Task 0 done
Task 3 done

This works as I expected. However, when I replace the trivial sleep task with a 'working' task, it doesn't work the same way...

import gevent
import random

def task(pid):
    start_time = time.time()
    array = []
    for x in range(0, 1000000):
        x = str(x) + "WasteMyTime"
        array.append(x)
    elapsed_time = time.time() - start_time
    print("Task " + str(pid) + " took  " + str(elapsed_time))

def synchronous():
    for i in range(1,10):
        task(i)

def asynchronous():
    threads = [gevent.spawn(task, i) for i in range(10)]
    gevent.joinall(threads)

print('Synchronous:')
synchronous()

print('Asynchronous:')
asynchronous()

Results

Synchronous:
Task 1 took  0.3065943717956543
Task 2 took  0.2897024154663086
Task 3 took  0.29267001152038574
Task 4 took  0.2936718463897705
Task 5 took  0.28526878356933594
Task 6 took  0.29134082794189453
Task 7 took  0.28323960304260254
Task 8 took  0.28522467613220215
Task 9 took  0.28423142433166504
Asynchronous:
Task 0 took  0.2896885871887207
Task 1 took  0.2837369441986084
Task 2 took  0.28224802017211914
Task 3 took  0.2857201099395752
Task 4 took  0.28621697425842285
Task 5 took  0.28621697425842285
Task 6 took  0.28621602058410645
Task 7 took  0.2857208251953125
Task 8 took  0.28720879554748535
Task 9 took  0.2847275733947754

The asynchronous tasks each slowly print out one after the other and the entire asynchronous block took just as long as the synchronous block. Is this user error?

Upvotes: 0

Views: 1370

Answers (2)

Nikhil M Jain
Nikhil M Jain

Reputation: 41

gevent is mostly used for I/O-bound operations. Your task is CPU-bound. Only one greenlet can be run at a time.

To see the impact of gevent, you should perform an I/O-bound task. Below is a modified example that makes a network request.

import time

import gevent
from gevent import monkey

monkey.patch_all()

import requests


def task(pid):
    start_time = time.time()
    requests.get('http://www.gstatic.com/generate_204')
    elapsed_time = time.time() - start_time
    print("Task " + str(pid) + " took  " + str(elapsed_time))


def synchronous():
    for i in range(1, 10):
        task(i)


def asynchronous():
    threads = [gevent.spawn(task, i) for i in range(10)]
    gevent.joinall(threads)


print('Synchronous:')
synchronous()

print('Asynchronous:')
asynchronous()

Upvotes: 0

dheeraj .A
dheeraj .A

Reputation: 1117

Add this on the top of your script(1st line of your script)

from gevent import monkey; monkey.patch_all()

Upvotes: 2

Related Questions