CODEWITHSUNDEEP
python
marie
marie

Reputation: 11

Download multiple dynamic webpages with BeautifulSoup

I'm trying to find a way to download multiple web pages that looks like this: https://domain.index.aspx?place=&time=123

How can I download each page that ends with a three digit number? I tried https://domain.index.aspx?place=&time=+[0-9] and '\d{3} but both do not work.

thanks

Upvotes: 1

Views: 293

Answers (1)

eyquem
eyquem

Reputation: 27575

You must know the ending numbers of the different pages you want to download and do, for exemple:

for numb in ('458', '123', '453'):
    sock = urllib.urlopen('https://domain.index.aspx?place=&time=' + numb)

or if you want to try all the numbers with 3 digits:

for numb in xrange(0,1000):
   sock = urllib.urlopen('https://domain.index.aspx?place=&time=' + str(numb).zfill(3))

Upvotes: 1

Related Questions