Reputation: 79
url_list = ['www.scrape.com/file1', 'www.scrape.com/file2', ''www.scrape.com/file3']
category_id_list = ['12345','abcde','ABCDE']
zip_list = ['10075','10017','10028']
I have three variables I use to create a URL to be requested. in the order: url_list+zip+categoryid
the url is then passed into a function which has the scrape code
I was using 3 for loops to iterate over these lists but that is highly redundant
for url_ in url_list:
for category_id in category_id_list:
for zip_ in zip_list:
request_url = url_+category_zip_
func(request_url)
This does the job, but is there a more optimal way to do it? Thank you!
Upvotes: 0
Views: 4725
Reputation: 23
One way to avoid writing multiple for loops is using zip.It allows you to access ith element from each list at once. So you can do something like:
url_list = ['www.scrape.com/file1', 'www.scrape.com/file2', 'www.scrape.com/file3']
category_id_list = ['12345','abcde','ABCDE']
zip_list = ['10075','10017','10028']
for url, id, zip in zip(url_list, category_id_list, zip_list):
request_url = url + id + zip
func(request_url)
Upvotes: 0
Reputation: 526
This might be a bit late, but this is the way I did it:
cats = ["a","b","c","d"]
zips = ["25320","53902","59607","53123"]
base = "https://example.com"
for i in range(4):
url = "{}/{}/{}".format(base, cats[i], zips[i])
print(url)
Output:
https://example.com/a/25320
https://example.com/b/53902
https://example.com/c/59607
https://example.com/d/53123
Upvotes: 0
Reputation: 11949
You may use itertools.product
import itertools
for url in (str.join("",url) for url in itertools.product(url_list,category_id_list,zip_list)):
func(url)
Upvotes: 3