CODEWITHSUNDEEP
rubyxpathnokogiri
Kane
Kane

Reputation: 924

Using Nokogiri to scrape a value from Yahoo Finance?

I wrote a simple script:

require 'rubygems'
require 'nokogiri'
require 'open-uri'

url = "http://au.finance.yahoo.com/q/bs?s=MYGN"
doc = Nokogiri::HTML(open(url))
name = doc.at_css("#yfi_rt_quote_summary h2").text
market_cap = doc.at_css("#yfs_j10_mygn").text
ebit = doc.at("//*[@id='yfncsumtab']/tbody/tr[2]/td/table[2]/tbody/tr/td/table/tbody/tr[11]/td[2]/strong").text


puts "#{name} - #{market_cap} - #{ebit}"

The script grabs three values from Yahoo finance. The problem is that the ebit XPath returns nil. The way I got the XPath was using the Chrome developer tools and copy and pasting.

This is the page I'm trying to get the value from http://au.finance.yahoo.com/q/bs?s=MYGN and the actual value is 483,992 in the total current assets row.

Any help would be appreciated, especially if there is a way to get this value with CSS selectors.

Upvotes: 0

Views: 459

Answers (2)

the Tin Man
the Tin Man

Reputation: 160581

Nokogiri supports:

require 'nokogiri'
require 'open-uri'

doc = Nokogiri::HTML(open("http://au.finance.yahoo.com/q/bs?s=MYGN"))
ebit = doc.at('strong:contains("Total Current Assets")').parent.next_sibling.text.gsub(/[^,\d]+/, '')

puts ebit
# >> 483,992

I'm using the <strong> tag as an place-marker with the :contains pseudo-class, then backing up to the containing <td>, moving to the next <td> and grabbing its text, then finally stripping the white-space using gsub(/[^,\d]+/, '') which removes everything that isn't a number or a comma.

Nokogiri supports a number of jQuery's JavaScript extensions, which is why :contains works.

Upvotes: 2

engineersmnky
engineersmnky

Reputation: 29498

This seems to work for me 

doc.css("table.yfnc_tabledata1 tr[11] td[2]").text.tr(",","").to_i
#=> 483992

Or as a string

doc.css("table.yfnc_tabledata1 tr[11] td[2]").text.strip.gsub(/\u00A0/,"")
#=> "483,992"

Upvotes: 0

Related Questions