CODEWITHSUNDEEP
pythonpython-requests
Squrppi
Squrppi

Reputation: 95

How to give Python requests headers to apply_async?

I'm working on parallel GET functionality with apply_async as described in Python Requests: Don't wait for request to finish. My problem is that I need to supply headers to every GET request and I fail to figure out how to do that.

I'm trying along these lines:

items.append(pool.apply_async(requests.get, [url, "", {"header1":"value1", "header2":"value2"}]))

and many variations of the theme without success.

I would appreciate information how to work my way through this.

Thanks!

Upvotes: 2

Views: 1150

Answers (1)

larsks
larsks

Reputation: 312018

According to the requests docs, you need to pass headers in the headers keyword argument to requests.get.

According to the multiprocessing docs, the arguments to apply_async are:

apply_async(func, args=(), kwds={}, callback=None)

Which in your case would translate to:

pool.apply_async(requests.get, 
                 [url], 
                 dict(headers={"header1":"value1", "header2":"value2"}))

Upvotes: 2

Related Questions