CODEWITHSUNDEEP
pythonpython-multiprocessingeventlet
Jacky Wang
Jacky Wang

Reputation: 3490

Why multi-process could not speedup sending HTTP requests via eventlet

I have an app to send a bunch of HTTP requests. First, I use eventlet and requests to implement it. But the performance is too low. Hence, I expected to speed it up using multi-process. A thing need to know is that the server will takes about 200ms to handle a single request (no including network transfer).

However, the multi-process is slower than the original version. I feel so amazed about this result! Why?

The code as following shown, I used timeit to measure time.

import eventlet
eventlet.monkey_patch(all=False, socket=True)

import requests

URL = 'http://....'

def send():
    pile = eventlet.GreenPile(20)
    for x in xrange(100):
        pile.spawn(requests.get, URL)
    for x in pile:
        pass

import multiprocessing

def main():
    procs = [multiprocessing.Process(target=send) for _ in range(3)]
    for p in procs:
        p.start()
    for p in procs:
        p.join()

import timeit

if __name__ == '__main__':
    print timeit.timeit(main, number=1)

Upvotes: 1

Views: 643

Answers (1)

temoto
temoto

Reputation: 5577

TL;DR: not enough information. By pure speculation, the server is the limiting factor (which may be caused by intentional artificial limits or resource starvation), so by adding more concurrent requests you are making each slower on average.

Here's one way to reason about this: you have limited amount of resources on both client and server: CPU cycles per time, memory accesses per time, memory size, network bandwidth. OS and eventlet make reasonable use of these resources. Such that you can do estimates on how much resources it takes to make a single request and software will scale it out in a reasonable pattern (that is close to linear). To benefit from multiprocessing would require your client process makes single CPU unit 100% busy. And specifically requests library is known to be good at wasting hardware resources, it incurs the most CPU overhead of all I tried (httplib, httplib2, urllib). But you have to make really lots (thousands) of concurrent requests or have really bad/busy CPU to make it bottleneck.

Exact answer requires information:

  • Whether HTTP client and server concur for any resources? I.e. do they run on same physical hardware?
  • What is the maximum request frequency (count per second) you were able to generate in single process mode? Adjust GreenPile size to vary number of concurrent requests.
  • What is the maximum frequency of requests you were able to generate using multiple processes? Adjust both GreenPile size and number of processes. Try running several independent Python interpreters without multiprocessing.
  • Was server the bottleneck? Check by adding another client on separate hardware. If request frequency is higher then the server is not bottleneck. If request frequency drops with more clients, then the server was already running at limit and multiprocessing could only help to make things worse.
  • What is the request time distribution? What percentage of requests was completed in 220/300/400/500/1000/more ms?
  • What is network latency/bandwidth? What is request/response size? Do you saturate network then?

Answering these questions will provide you with excellent intuition on what's going on inside and between your client/server.

Relevant Github issue: https://github.com/eventlet/eventlet/issues/195

Upvotes: 1

Related Questions