CODEWITHSUNDEEP
python-3.xgetpython-requests
Schalton
Schalton

Reputation: 3104

Python3 get.requests url missing params

I'm running python 3.6 on windows 10 -- standard install, jupyter notebooks IDE

Code:

import requests

params={'q': 'NASDAQ:AAPL','expd': 19, 'expm': 1, 'expy': 2018, 'output': 'json'}
response = requests.get('https://www.google.com/finance/option_chain', params=params)

print(response.url)

Expected Output:

https://finance.google.com/finance/option_chain?q=NASDAQ:AAPL&expd=19&expm=1&expy=2018&output=json

Actual Output:

https://finance.google.com/finance/option_chain?q=NASDAQ:AAPL&output=json

Thank you for taking a look at my code!

-E

Upvotes: 2

Views: 4094

Answers (2)

Gao Liang
Gao Liang

Reputation: 66

Because you get a URL redirection. HTTP status code is 302 and you were redirected to a new url.

You can get more info here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections and http://docs.python-requests.org/en/master/user/quickstart/#redirection-and-history

We can use the history property of the Response object to track redirection. try this:

import requests

params={'q': 'NASDAQ:AAPL','expd': 19, 'expm': 1, 'expy': 2018, 'output': 
'json'}
response = requests.get('https://www.google.com/finance/option_chain', 
params=params)
print(response.history) #  use the history property of the Response object to track redirection.
print(response.history[0].url) # print the redirect history's url
print(response.url)

You will get:

[<Response [302]>]
https://www.google.com/finance/option_chain?q=NASDAQ%3AAAPL&expm=1&output=json&expy=2018&expd=19
https://finance.google.com/finance/option_chain?q=NASDAQ:AAPL&output=json

You can disable redirection handling with the allow_redirects parameter:

import requests

params={'q': 'NASDAQ:AAPL','expd': 19, 'expm': 1, 'expy': 2018, 'output': 'json'}
response = requests.get('https://www.google.com/finance/option_chain', params=params, allow_redirects=False)
print(response.status_code)
print(response.url)

Upvotes: 2

Schalton
Schalton

Reputation: 3104

This isn't really an answer, but it's my workaround using string concat

    response = requests.get(url, params=params)
    response_url = response.url
    added_param = False
    for i in params:
        if response_url.find(i)==-1:
            added_param = True
            response_url = response_url+"&"+str(i)+"="+str(params[i])
            print("added:",str(i)+"="+str(params[i]), str(response_url))
    if added_param:
        response = requests.get(response_url)

Upvotes: 0

Related Questions