CODEWITHSUNDEEP
pythonscrapy
John Rogerson
John Rogerson

Reputation: 1183

Scrapy Can't Select By ID

I'm trying to scrape data for a school project but for some reason I can't figure out how to gt the data from the table on this page. I'm testing out in scrapy shell and just not getting any data back. Here's what I'm trying

scrapy shell https://www.psacard.com/pop/baseball-cards/1911/turkey-red-cabinets-t3-t9/51453

table = response.xpath('//*[@id="DataTables_Table_0"]/table/tbody')

this is returning []

I've tried with the entire table class as well like such

table = response.xpath('//*[@class="clear interior-result-table pop-grid items psa table-striped dataTable no-footer dtr-inline"]')

and getting the same [] response

i'm stumped. any ideas?

Upvotes: 1

Views: 130

Answers (1)

vezunchik
vezunchik

Reputation: 3717

All data that you need is in another request. You can find it in developer tools in Chrome, for example. Check https://www.psacard.com/Pop/GetItemTable?headingID=51453&categoryID=20003&isPSADNA=false&pf=0&_=1555080293549. Even common GET request to this link will give you table rows.

Upvotes: 3

Related Questions