CODEWITHSUNDEEP
pythondjangomultithreadingrest
brain storm
brain storm

Reputation: 31272

submitting multiple POST requests as multi threads in Python/Django

My development stack is Django/Python. (that includes Django-REST-framework) I am currently looking for ways to do multiple distinct API calls.

client.py

def submit_order(list_of_orders):
   //submit each element in list_of_orders in thread
   for order in list_of_orders:
      try:
        response=request.POST(order, "www.confidential.url.com")
      except:
         //retry_again
      else:
          if !response.status==200:
              //retry_again

In the above method, I am currently submitting order one by one, I want to submit all orders at once. Secondly, I want to retry submission for x times if it fails. I currently do not know how well to achieve it.

I am looking for ways that python libraries or Django application provide rather than re-inventing the wheel.

Thanks

Upvotes: 1

Views: 1653

Answers (1)

avenet
avenet

Reputation: 3043

As @Selcuk said you can try django-celery which is a recommended approach in my opinion, but you will need to make some configuration and read some manuals.

On the other hand, you can try using multiprocessing like this:

from multiprocessing import Pool

def process_order(order):
    #Handle each order here doing requests.post and then retrying if neccesary 
    pass

def submit_order(list_of_orders):
    orders_pool = Pool(len(list_of_orders))
    results = orders_pool.map(process_order, list_of_orders)
    #Do something with the results here

It will depend on what you need to get done, if you can do the requests operations on the background and your api user can be notified later, just use django-celery and then notify the user accordingly, but if you want a simple approach to react immediately, you can use the one I "prototyped" for you.

You should consider some kind of delay on the responses for your requests (as you are doing some POST request). So make sure your POST request don't grow a lot, because it could affect the experience of the API clients calling your services.

Upvotes: 2

Related Questions