CODEWITHSUNDEEP
htmlpython-2.7xpathweb-scrapingscrapy
loremIpsum1771
loremIpsum1771

Reputation: 2527

xpath finds element in developers console but not in scrapy.response

I'm trying to scrape the price from the first ticket here page using this xpath:

'.//*[@class="price"]/text()'

This works in the developer's console, but not when I run it in the scrapy shell using response.xpath. I have also tried to following in the shell:

'.//*[@class="initial"]/div[@class="price"]/text()'

and

'//*[@id="tVB901769989"]/div[1]/div[4]' (although I don't think that the id property can be used in the shell like this).

Is there something wrong with the xpaths that I've used, or is there some thing different with the way the page works? Any help would be appreciated. Thanks!

Upvotes: 0

Views: 138

Answers (1)

eLRuLL
eLRuLL

Reputation: 18799

this happens because you are checking at different requests, the page you see doesn't have the information you need inside that file, but it gets it dynamically, in this case from: www.vividseats.com/javascript/tickets.shtml?productionId=1771684

There you can check the prices on json format, I think this is for one item:

{
   "s":"Section 111",
   "r":"8-22",
   "q":"4",
   "p":"692.00",
   "i":"VB782041491",
   "d":"111",
   "n":"Zone Seating. The seller is committing to procure these tickets for you upon receipt of your order. After you place your order and your order is confirmed, we guarantee that your tickets will be within the listed zone
 or section listed or one comparable and that you will receive these tickets in time for the event or
 your money back. Orders exceeding four tickets may be split up into different rows within the requested
 zone or section.",
   "f":"0",
   "l":"Section 111",
   "g":"0",
   "e":"0",
   "h":"07/21/15",
   "t":"0",
   "v":"",
   "c":"84352",
   "z":"1",
   "rhdn":"0",
   "ind":"0",
   "sd":"0"
}

where p contains the price.

Upvotes: 3

Related Questions