CODEWITHSUNDEEP
htmlxpathweb-scraping
Sachin Kainth
Sachin Kainth

Reputation: 46740

Why is XPath failing to select anything from HTML?

My HTML looks like

...
<lots of nested tags>
  <div class="tiles-tile">
    <div class="brick">
      <div class="brick-header">
        ...
        <ul class="links">
          <li><i class="icon-minus"></i></li>
          <li><i class="icon-resize-small"></i></li>
          <li><i class="icon-list"></i></li>
          <li><i class="icon-time"></i></li>
          <li><a></a></li>
        </ul>
      </div>
    </div>
  </div>
</lots of nested tags>
<lots of nested tags>
  <div class="tiles-tile">
    <div class="brick">
      <div class="brick-header">
        ...
        <ul class="links">
          <li><i class="icon-minus"></i></li>
          <li><i class="icon-resize-small"></i></li>
          <li><i class="icon-list"></i></li>
          <li><i class="icon-time"></i></li>
          <li><a></a></li>
        </ul>
      </div>
    </div>
  </div>
</lots of nested tags>
...

I've tried everything I can think of, by using XPath, to get a list of all <i class="icon-list" elements but I just cannot figure it out.

For instance I tried

//div[@class=\"brick-header\"]/ul/li/i[@class=\"icon-list\"]

but this gives me an empty list.

How do I do this?

Upvotes: 0

Views: 84

Answers (1)

kjhughes
kjhughes

Reputation: 111521

The following XPath will select all i elements of class icon-list regardless of ancestral elements:

//i[@class='icon-list']

Reasons your XPath might not be working

  1. There is a namespace declaration to take into account.

    Remedy: Declare proper namespace prefix and use it in the XPath (preferable), or test against the local name to avoid any namespace:

    //*[local-name()='i' and @class='icon-list']

  2. There are other classes listed along with icon-list.

    Remedy: Use contains() rather than equality in the predicate:

    //i[contains(concat(' ',@class,' '), ' icon-list ')]

  3. JavaScript is building or changing the markup structure after your XPath is running.

    Remedy: Wait to apply test until document has finished loading and JavaScript has finished running.

Upvotes: 2

Related Questions