CODEWITHSUNDEEP
pythonipproxies
Jakub
Jakub

Reputation: 1710

Rotating Proxy IPs Before accessing API to get json request using Python

Question 1 How do I use an new IP Address before the API Request is made?

Question 2 Is there a good way to test if this IP thing actually worked other than printing results of grabbed IP

I made a new file with list of IPs in each line and I grab the proxy using

 proxies_lines = open('proxies').read().splitlines()
 proxy=random.choice(proxies_lines)

I tried two ways to store the proxies

 https://000.00.000.00:0000
 https://000.00.000.00:0000
 https://000.00.000.00:0000
 
 Also saw some posts of people storing them this way. Not sure which way is best?

 000.00.000.00:0000
 000.00.000.00:0000
 000.00.000.00:0000

What I want to do with this code

  1. Iterate through my list of 7 stocks in stock_list
  2. Grab a new IP address
  3. Query the API with the stock ticker, API Key and new IP address
  4. Print out the results for the stocks.

I have all steps completed but #3 regarding adding the IP address.

This is how I have the for loop structured

for item in stock_list:
    stock_ticker=item
    keys=random.choice(lines)
    proxy=random.choice(proxies_lines)
    time.sleep(1)

    # To access the API
    base_url = 'https://www.alphavantage.co/query?'
    params = {'proxies' : proxy,
             'function': 'OVERVIEW',
             'symbol': stock_ticker,
             'apikey': keys}
    
    response_data_overview = requests.get(base_url, params=params)
    data_overview_MarketCapitalization = response_data_overview.json()['MarketCapitalization']
           
    # Print Results
    print("The Market Cap for {} is = {}".format(stock_ticker,data_overview_MarketCapitalization))
    print("Proxie Used {}".format(proxy))

Is this the correct way of setting things up? When I run the code it prints out the market cap for 5 of the 7 stocks before stopping. If the IP rotation would work, than it would print out 7 / 7 results since I can only do 5 requests per minute for one IP address / key.

Side Rant

Apologizes in advance if this is a super nooby question. Learning coding and decided to go with Python as the first language last month. While taking Python classes on team treehouse trying to do a side project of building an open source stock screener with pre-built well established valuation formulas. Super early stages lol but progress is progress https://github.com/Jakub-MFP/FIRE_Dashboard

I'm a future Pythonista in training haha

Upvotes: 0

Views: 793

Answers (1)

Jiří Baum
Jiří Baum

Reputation: 6930

  • Looks like you're putting the proxy into the params, which is the data that gets posted to the server; instead, you need to pass it to requests.get() so that it knows how to make the request.

     params = {'function': 'OVERVIEW',
           'symbol': stock_ticker,
           'apikey': keys}

 response_data_overview = requests.get(base_url,
                                       params=params,
                                       proxies={'https': proxy})

  • Circumventing posted API limits is rude at best and a felony at worst. You should either make the requests at a slower rate (ie, less than 5 per minute) or get a premium plan with higher limits (or other negotiated, probably paid access).

    If this is a learning project, making the requests at a slower rate is probably the best approach.

Upvotes: 1

Related Questions