CODEWITHSUNDEEP
python-3.xpython-requestspython-asynciopython-multithreading
Rafael Lobo
Rafael Lobo

Reputation: 45

Python 3 threading post request passing header params and data

I'm trying to make my post requests faster because at the moment it takes 3 seconds per post. As I need to iterate it n times it could take hours. So, I started looking for threading, async calls and many others but none solved my problem. Mostly of the problems was due to the fact that I couldn't specify the headers and the params of my post request.

My Python version is 3.6.7

My code:

for i in range(0, 1000):
  assetId = jsonAssets[i]['id']
  uuidValue = uuid.uuid4()

  headers = {'Content-Type': 'application/json',}
  params = (('remember_token', '123456'),)

  data = ('{{"asset":'
          '{{"template":1,'
          '"uuid":"{uuidValue}", '
          '"assetid":{assetId}}}}}'
          .format(uuidValue = uuidValue, 
                  assetId = assetId))
  response = requests.post('http://localhost:3000/api/v1/assets', headers=headers, params=params, data=data)

Some of the tries were using:

pool.apply_async

or

ThreadResponse

But I couldn't set headers or params like in the request.post

So, how can I make this post request using this header, params and data faster?

Thanks in advance and sorry for any trouble, this is my first stackoverflow post.

Upvotes: 2

Views: 1014

Answers (2)

Mikhail Gerasimov
Mikhail Gerasimov

Reputation: 39606

If you can make single request properly, shortest way for you is to use ThreadPoolExecutor:

def single_request(i):
  assetId = jsonAssets[i]['id']
  uuidValue = uuid.uuid4()
  # ... all other requests stuff here

  return response


with ThreadPoolExecutor(max_workers=10) as executor:
    futures = {
        executor.submit(single_request, i): i
        for i 
        in range(1000)
    }

    for future in as_completed(futures):
        i = futures[future]
        try:
            res = future.result()
        except Exception as exc:
            print(f'excepiton in {i}: {exc}')
        else:
            print(res.text)

Upvotes: 1

ipave
ipave

Reputation: 1338

You should use async libraries like aiohttp

Upvotes: 0

Related Questions