Send multiple Web requests

Question

Im trying to send API requests at specific rate (Example using QPS value), I'm using the following code to launch my requests (stored in web_requests list).

If I set QPS=10, delay will be 0.1s or 100 ms. I use time.sleep(0.1) and send a request, but this code is waiting for remote end HTTP response which is around 30 ms, so I end up having 0.3 seconds of extra delay total. How can I send X web requests per seconds without waiting for the response?

@gen.coroutine
def send_requests(campaign_instance):
    ...
    http_client = httpclient.AsyncHTTPClient()
    while True:
                try:                        
                    web_request = web_requests.pop()
                    time.sleep(delay)
                    headers = {'Content-Type': 'application/json'}
                    request = httpclient.HTTPRequest(auth_username=settings.api_account,
                                                     auth_password=settings.api_password,
                                                     url=settings.api_web_request,
                                                     body=json.dumps(web_request),
                                                     headers=headers,
                                                     request_timeout=5,
                                                     method="POST")
                    yield http_client.fetch(request, callback=partial(handle_response, web_request["to"]))

                    gen_log.info("start_campaign() Requests in Queue: {}".format(len(web_requests)))

                except httpclient.HTTPError, exception:
                    gen_log.info.exception("start_campaign() ".format(exception))
                    api_errors += 1
                    if handle_api_errors(api_errors):
                        break
                except IndexError:
                    gen_log.info.info('start_campaign() Campaign web requests completed. API Errors: {}'.format(api_errors))
                    break

def start():
    ioloop.IOLoop.current().run_sync(lambda: send_requests(campaign_instance))
    log.info('process_campaign() Campaign completed')
    campaign_instance.terminate()

Answer 1

Simply do not "yield" the future returned by "fetch". Then your coroutine will continue looping immediately, and the callback is executed when the fetch completes in the background.

Also, never ever ever call "sleep" in a Tornado application:

http://www.tornadoweb.org/en/stable/faq.html#why-isn-t-this-example-with-time-sleep-running-in-parallel

If you do, all processing stops and your "fetch" is hung until the sleep completes. Instead:

yield gen.sleep(delay)