Reputation: 183
I am trying to scrape car data. Their 'id' tags increment by 1, however i just can't seem to figure out how to do that. Here is what I have:
import bs4 as bs
import urllib
source = urllib.request.urlopen('http://www.25thstauto.com/inventory.aspx?cursort=asc&pagesize=500').read()
soup = bs.BeautifulSoup(source, 'lxml')
#finds the total number of cars
count = soup.find('span', {'id': 'ctl00_cphBody_inv1_lblVehicleCount'}).getText()[:2]
count = int(count)
i = 1
for url in range(1,count):
url = soup.find_all('a', {'id': 'ctl00_cphBody_inv1_rptInventoryNew_ctl0'+i+'_nlVehicleDetailsTitle'})
print(url['href'])
i = i + 1
Upvotes: 1
Views: 223
Reputation: 12178
import bs4 as bs
import urllib
import re
source = urllib.request.urlopen('http://www.25thstauto.com/inventory.aspx?cursort=asc&pagesize=500').read()
soup = bs.BeautifulSoup(source, 'lxml')
for a in soup.find_all('a', id=re.compile('ctl00_cphBody_inv1_rptInventoryNew')):
print(a.get('href'))
out:
2008_Chevrolet_Malibu_Easton_PA_265928462.veh
2008_Chevrolet_Malibu_Easton_PA_265928462.veh
2008_Chevrolet_Malibu_Easton_PA_265928462.veh
2002_Nissan_Xterra_Easton_PA_266894015.veh
2002_Nissan_Xterra_Easton_PA_266894015.veh
2002_Nissan_Xterra_Easton_PA_266894015.veh
2009_Chevrolet_Cobalt_Easton_PA_265621796.veh
2009_Chevrolet_Cobalt_Easton_PA_265621796.veh
Use regex to find a
tag whose id
attribute contain ctl00_cphBody_inv1_rptInventoryNew
OR use CSS selector:
for a in soup.select('a[id*=ctl00_cphBody_inv1_rptInventoryNew]'):
print(a.get('href'))
The idea is same.
Upvotes: 1