Reputation: 473
I'm trying to scrape some data off a site using proxies, I won't put any code here since it is a bit long and maybe not important for the question. I can just list these facts:
requests.session()
and httpx.Client()
, and both are failing in the same way.
So I think the only conclusion here is that my IP is somehow getting leaked by these proxies and shown to the site, triggering a block from it. Proxies in question are Brightdata Datacenter (cheap ones) and Residential proxies from proxy-cheap.com which should be a bit better.
Is there a way to prevent my IP from being sent over with proxies? I don't really think there is a different option, how else could they block rotating proxies from different providers, after letting them work initially for a while.
And by the way, I'm sure it's not some mistake by me, session is created like this
session = httpx.Client(proxies={"http://":proxy_url, "https://":proxy_url})
So that every request uses the same proxy.
Upvotes: 0
Views: 367
Reputation: 1
I believe the easy solution would be buying an elite proxy, since some proxy will tell the server that request is made by a proxy.
Upvotes: 0