CODEWITHSUNDEEP
htmlrubynokogiri
John Feminella
John Feminella

Reputation: 311526

Extract links not inside parentheses from Nokogiri document

I have a Nokogiri::HTML document. It corresponds to content from a Wikipedia article that might look like this:

James Henry 'Jimmie' Lyons (born in Chicago, Illinois – November 6, 1892 – October 10, 1963) was a baseball player in the Negro Leagues. He pitched and played outfield and between 1910 to 1925.

which has the corresponding HTML:

<p><b>James Henry 'Jimmie' Lyons</b> (born in <a href="/wiki/Chicago,_Illinois" title="Chicago, Illinois" class="mw-redirect">Chicago, Illinois</a> – November 6, 1892 – October 10, 1963) was a <a href="/wiki/Baseball" title="Baseball">baseball</a> player in the <a href="/wiki/Negro_League_baseball" title="Negro League baseball" class="mw-redirect">Negro Leagues</a>.<sup id="cite_ref-5" class="reference"><a href="#cite_note-5"><span>[</span>5<span>]</span></a></sup> He <a href="/wiki/Pitcher" title="Pitcher">pitched</a> and played <a href="/wiki/Outfielder" title="Outfielder">outfield</a> and between 1910 to 1925.

I'd like to extract the value of the href attribute of the first non-parenthesized <a> tag in this document.

In this case, the correct answer is to extract "/wiki/Baseball", the href attribute of the second link, because the first link's href, /wiki/Chicago,_Illinois, is inside of parentheses.

Note that <a> tags can themselves contain parentheses in their hrefs, so a naive approach like "strip all the parentheses from the HTML" is not correct.

What's the best way to do that? I am pretty sure I'm going to need to use Nokogiri's SAX parser but if there's an easier way I'd love that.

Upvotes: 2

Views: 106

Answers (1)

Justin Ko
Justin Ko

Reputation: 46836

You could try taking the first link where the preceding text nodes have the same number of opening and closing parenthesis.

require 'nokogiri'

def first_non_parenthesized_href(html)
    doc = Nokogiri::HTML(html)
    return doc.css('a').find{ |a|
        previous_text = a.xpath('preceding::text()').collect(&:text).join
        previous_text.count('(') == previous_text.count(')')
    }['href']   
end

# Original example
html = %q{<p><b>James Henry 'Jimmie' Lyons</b> (born in <a href="/wiki/Chicago,_Illinois" title="Chicago, Illinois" class="mw-redirect">Chicago, Illinois</a> - November 6, 1892 - October 10, 1963) was a <a href="/wiki/Baseball" title="Baseball">baseball</a> player in the <a href="/wiki/Negro_League_baseball" title="Negro League baseball" class="mw-redirect">Negro Leagues</a>.<sup id="cite_ref-5" class="reference"><a href="#cite_note-5"><span>[</span>5<span>]</span></a></sup> He <a href="/wiki/Pitcher" title="Pitcher">pitched</a> and played <a href="/wiki/Outfielder" title="Outfielder">outfield</a> and between 1910 to 1925.}
puts first_non_parenthesized_href(html)
#=> "/wiki/Baseball"

# Example in comment
html = %q{<p><b>Science</b> (from <a href="/wiki/Latin_language" title="Latin language" class="mw-redirect">Latin</a> <i>scientia</i>, meaning "knowledge"<sup id="cite_ref-OnlineEtDict_1-0" class="reference"><a href="#cite_note-OnlineEtDict-1"><span>[</span>1<span>]</span></a></sup>) is a systematic enterprise that builds and organizes <a href="/wiki/Knowledge" title="Knowledge">knowledge</a> in the form of testable explanations and predictions about the <a href="/wiki/Universe" title="Universe">universe</a>.<sup id="cite_ref-wilson_2-0" class="reference"><a href="#cite_note-wilson-2"><span>[</span>2<span>]</span></a></sup><sup id="cite_ref-3" class="reference"><a href="#cite_note-3"><span>[</span>3<span>]</span></a></sup> In an older and closely related meaning, "science" also refers to a body of knowledge itself, of the type that can be rationally explained and reliably applied. A practitioner of science is known as a <a href="/wiki/Scientist" title="Scientist">scientist</a>.</p>}
puts first_non_parenthesized_href(html)
#=> "/wiki/Knowledge"

Upvotes: 1

Related Questions