CODEWITHSUNDEEP
pythonpython-requests
miran80
miran80

Reputation: 1105

How many http requests do I make, when I request the document page with Requests in Python 3.9?

I am using a python script to parse a website via proxy. Said proxy implements a limit on how many concurrent requests can I make. When running a single thread of my python application, I encountered my proxy provider rate limiting me and I wonder, if I am really making more than 1 request when I run requests.get(url, headers=headers, proxies=proxies) Shouldn’t this only return the document page without any other resources?

Upvotes: 0

Views: 117

Answers (1)

Skarlinski
Skarlinski

Reputation: 2479

As tested below, a single request is sent. Maybe what is happening is that your proxy provider has some mechanism to detect bots, and is blocking your script.

This mechanism could use anything from user agent, to how the packets are sent. python on left, node server on right

Upvotes: 2

Related Questions