CODEWITHSUNDEEP
python
user2205861
user2205861

Reputation: 11

Changing variable for parameters for http json get requests

Pretty new to python so go easy on me :). This code works below but I was wondering if there is a way to change the indcode parameter by doing a loop so I do not have to repeat the requests.get.

paraD = dict()
paraD["area"] = "123"
paraD["periodtype"] = "2"
paraD["indcode"] = "722"
paraD["$limit"]=1000

#Open URL and get data for business indcode 722
document_1 = requests.get(dataURL, params=paraD)
bizdata_1 = document_1.json()


#Open URL and get data for business indcode 445
paraD["indcode"] = "445"
document_2 = requests.get(dataURL, params=paraD)
bizdata_2 = document_2.json()


#Open URL and get data for business indcode 311
paraD["indcode"] = "311"
document_3 = requests.get(dataURL, params=paraD)
bizdata_3 = document_3.json()

#Combine the three lists
output = bizdata_1 + bizdata_2 + bizdata_3

Upvotes: 1

Views: 423

Answers (2)

Adon Bilivit
Adon Bilivit

Reputation: 26825

Assuming you're using Python 3.9+ you can combine dictionaries using the | operator. However, you need to be sure that you understand exactly what this will do. It's more likely that the code to combine dictionaries will be more complex.

When using the requests module it is very important to check the HTTP status code returned from the function (HTTP verb) you're calling.

Here's an approach to the stated problem that may work (depending on how the dictionary merge is effected).

from urllib.error import HTTPError
from requests import get as GET
from requests.exceptions import Timeout, TooManyRedirects, RequestException
from sys import stderr

# base parameters
params = {'area': '123', 'periodtype': '2', '$limit': 1000}
# the indcodes 
indcodes = ('722', '445', '311')

# gets the JSON response (as a Python dictionary)
def getjson(url, params):
    try:
        (r := GET(url, params, timeout=1.0)).raise_for_status()
        return r.json() # all good
    # if we get any of these exceptions, report to stderr and return an empty dictionary
    except (HTTPError, ConnectionError, Timeout, TooManyRedirects, RequestException) as e:
        print(e, file=stderr)
        return {}
    # any exception here is not associated with requests/urllib. Report and raise
    except Exception as f:
        print(f, file=stderr)
        raise

# an empty dictionary
target = {}

# build the target dictionary
# May not produce desired results depending on how the dictionary merge should be carried out
for indcode in indcodes:
    target |= getjson('https://httpbin.org/json', params | {'indcode' : indcode})

Upvotes: 0

Ananth
Ananth

Reputation: 4397

Since indcode is the only parameter that changes for each request, we will put that in a list and make the web requests inside a loop.

data_url = ""

post_params = dict()
post_params["area"] = "123"
post_params["periodtype"] = "2"
post_params["$limit"]=1000


# The list of indcode values
ind_codes = ["722", "445", "311"]

output = []

# Loop on indcode values
for code in ind_codes:
    # Change indcode parameter value in the loop
    post_params["indcode"] = code

    try:
        response = requests.get(data_url, params=post_params)
        data1 = response.json()
        output.append(data1)
    except:
        print("web request failed")
        # More error handling / retry if required

print(output)

Upvotes: 2

Related Questions