CODEWITHSUNDEEP
ruby-on-railsrubyxmlxpathnokogiri
chintanparikh
chintanparikh

Reputation: 1682

Nokogiri not working for advanced XPath search

Take a look at this page: http://finviz.com/quote.ashx?t=AAPL

The XPath 

(//*[contains(concat(' ', @class, ' '), ' snapshot-table2 ')]/tbody/tr/td[text() = 'Index']/following::td)[1]/b

should spit out S&P 500, and when I try it out in the JavaScript console, it does. However, when I try using Nokogiri with Ruby, it returns an empty array.

Here's the full code:

url = "http://finviz.com/quote.ashx?t=#{ticker}"
data = Nokogiri::HTML(open(url))
xpath = "(//*[contains(concat(' ', @class, ' '), ' snapshot-table2 ')]/tbody/tr/td[text() = '#{datapoint}']/following::td)[1]/b"
data.xpath(xpath)

data.xpath(xpath) returns []

Any ideas?

Upvotes: 0

Views: 288

Answers (1)

alol
alol

Reputation: 1152

In the response of that page, I don't see a <tbody> element.

You mentioned your XPath works in your browser console, this is likely because the browser is injecting the tbody element for rendering purposes, see "Why do browsers insert tbody element into table elements?" for detail on why this happens.

Try your XPath again without specifying the <tbody> node:

data.xpath("(//*[contains(@class, 'snapshot-table2')]/tr/td[text() = 'Index']/following::td)[1]/b")

You could also use something like this to get the same XPath to work in both the browser and your Ruby code:

(//*[contains(@class, 'snapshot-table2')]//tr/td[text() = 'Index']/following::td)[1]/b

Upvotes: 2

Related Questions