CODEWITHSUNDEEP
pythonhttpsproxypython-requests
Synaps
Synaps

Reputation: 323

How to send GET requests via python over different IPs?

I am currently trying to gather data from an exchange for deep learning. The issue is I need the data with a high resolution (second intervals), therefore I am creating a lot of GET requests for each currency. But I reach fast the limits of the API really.

I would like to know if there is a possibility to route the GET request over a public proxy server to claim to have a different IP-address to make as many requests as possible with a Raspberry PI.

Upvotes: 1

Views: 5277

Answers (2)

Hamza
Hamza

Reputation: 133

This is just an more detailed version of the verified answers

How To Download?

pip3 install proxy-requests

Example for Basic GET Request

from proxy_requests import ProxyRequests
r = ProxyRequests("https://dog.ceo/api/breeds/image/random")
r.get()

Example for GET Request with Headers

from proxy_requests import ProxyRequests
h = {'User-Agent': 'NCSA Mosaic/3.0 (Windows 95)'}                                                             
r = ProxyRequests('url here')                                                                                  
r.set_headers(h)                                                                                               
r.get_with_headers()

Example for POST Request

from proxy_requests import ProxyRequests
r = ProxyRequests('url here')                                                                                  
r.post({'key1': 'value1', 'key2': 'value2'})

Example for POST Request with Headers

r = ProxyRequests('url here')
r.set_headers({'name': 'rootVIII', 'secret_message': '7Yufs9KIfj33d'})
r.post_with_headers({'key1': 'value1', 'key2': 'value2'})

Example for Request with Authentication

r = ProxyRequestsBasicAuth('url here', 'username', 'password')
r.get()

By The way, all these thing are already mentioned on the Github repository.

Upvotes: 1

user10364045
user10364045

Reputation:

If you need to make a GET with different proxies/IP addresses...

Try proxy-requests. I am not sure if that's what you are asking, but it seems like maybe you reached your request limit to a your API endpoint. Some organizations limit the number of requests allowed from certain IP addresses.

Here's an example GET using python proxy requests so that your IP will be different:

pip3 install proxy-requests

  

from proxy_requests.proxy_requests import ProxyRequests
r = ProxyRequests("https://api.ipify.org")
r.get()

pypi

github

The module seems to use scraped proxies to make the request. I just tried quickly and it worked

Upvotes: 5

Related Questions