CODEWITHSUNDEEP
pythonmultithreadingapipython-requests
šeki
šeki

Reputation: 153

MultiThreading python post request

I need help with multithreading.

I had no experience with that and I don't know how accurate it is. I made a post request for api and get a response every 1 second. I saw on the internet that it is possible to send multiple requests at once and get multiple replies (sorry if I'm wrong)

This is a post request:

url = 'https://open-api.trovo.live/openplatform/channels/id'
payload = {"channel_id":100379073}
headers = {'Client-ID': '#######', 'Accept-Charset': 'UTF-8'}
r = requests.post(url, data=payload, headers=headers)

Max QPS is 20, i want to send 15 request in one second, if someone can tell mi if this is possible to make?

I want to make something like this:

url = 'https://open-api.trovo.live/openplatform/channels/id'
users = ['1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234','1234']
headers = {'Client-ID': '#######', 'Accept-Charset': 'UTF-8'}
for users in users:
    payload = {"channel_id":users}
    r = requests.post(url, data=payload, headers=headers)

Upvotes: 0

Views: 1308

Answers (1)

xcapt
xcapt

Reputation: 84

For GET or POST asynchronous requests in python, you should use concurrent.futures library. Here is an example

max_workers is a parameter that depends on your computer

import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import time

# ['1234', '1234', ...] 20 times
users = ['1234'] * 20

def make_request(user):
    url = 'https://open-api.trovo.live/openplatform/channels/id'
    headers = {'Client-ID': '#######', 'Accept-Charset': 'UTF-8'}
    payload = {"channel_id": user}
    return requests.post(url, data=payload, headers=headers)


# standard request
start = time.time()    
for user in users:
    make_request(user)
print(time.time() - start, 's')

# Output: 
# 4.139220714569092s



# asynchronous
start = time.time()  
processes = []
with ThreadPoolExecutor(max_workers=10) as executor:
    for user in users:
        processes.append(executor.submit(make_request, user))
        
for task in as_completed(processes):
    print(task.result())
print(time.time() - start, 's')

# Output:
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# <Response [500]>
# 0.499767541885376s

Upvotes: 1

Related Questions