CODEWITHSUNDEEP
javascripthtmlweb-scraping
L.Goyal
L.Goyal

Reputation: 895

Not able to parse HTMLCollection list

I am trying to scrape out some links from a site using javascript. In the below HTML I have some links in the 'a' tag. And I am using the following JS.

The problem is that I am not able to get the links from the myChildList.

When I console.log myDownloadDiv, it shows me the output as object HTMLElement. When I console.log the myChildList (i.e the children of the HTMLElement) I get something like a HTMLCollection [a.dropdown-item.disabled] in which the length is 2 and with both the anchor tags and with all their properties in it with the links.

When I want to console.log all the elements using a loop, it shows only one element and the output is like this:

<a class="dropdown-item disabled"></a>

I don't understand if I am doing something wrong while parsing out the HTMLCollection List.

var myDownloadDiv = document.getElementById("pickDownload");
var myChildList = myDownloadDiv.children;
console.log(myChildList);

for (var j of myChildList) {
  console.log(j);
}
<div class="col-12 col-sm-3">
  <div class="dropup">
    <a href="" class="btn btn-secondary btn-block dropdown-toggle" id="downloadMenu" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
            Download
            </a>

    <div class="dropdown-menu" id="pickDownload" aria-labelledby="downloadMenu">
      <a href="https://fainbory.com/4eQM" class="dropdown-item" target="_blank">
        <span class="badge badge-primary">BD</span> Elysium - 720p (129 MB)
      </a>
      <a href="https://fainbory.com/4eQP" class="dropdown-item" target="_blank">
        <span class="badge badge-primary">BD</span> Elysium - 1080p (239 MB)
      </a>
    </div>
  </div>
</div>

EDIT:

I tried the solutions. They work perfectly in the snippets but they are not working on the site which I am trying to scrape.

Upvotes: 0

Views: 151

Answers (2)

Simone Borda
Simone Borda

Reputation: 1

your code is correct and works properly on my machine, I think there's an issue with your console object.

Please try to remove this import https://static.lalaping.com/online.js?ver=2.0.0:formatted I suspect the console object is overwritten there.

You can verify this on you site, just running in the browser console the following commands:

// Restore console object:
var i = document.createElement('iframe');
i.style.display = 'none';
document.body.appendChild(i);
window.console = i.contentWindow.console;

// Run your code:
var myDownloadDiv = document.getElementById("pickDownload");
var myChildList = myDownloadDiv.children;
console.log(myChildList);

for (var j of myChildList) {
  console.log(j);
}

Upvotes: 0

mplungjan
mplungjan

Reputation: 177955

Your loop seems to work.

Perhaps you meant this

const links = [...document.querySelectorAll("#pickDownload a")].map(link => link.href);
console.log(links);
<div class="col-12 col-sm-3">
  <div class="dropup">
    <a href="" class="btn btn-secondary btn-block dropdown-toggle" id="downloadMenu" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
            Download
            </a>

    <div class="dropdown-menu" id="pickDownload" aria-labelledby="downloadMenu">
      <a href="https://fainbory.com/4eQM" class="dropdown-item" target="_blank">
        <span class="badge badge-primary">BD</span> Elysium - 720p (129 MB)
      </a>
      <a href="https://fainbory.com/4eQP" class="dropdown-item" target="_blank">
        <span class="badge badge-primary">BD</span> Elysium - 1080p (239 MB)
      </a>
    </div>
  </div>
</div>

Here is a bookmarklet

javascript:(function() { const links = [...document.querySelectorAll("#pickDownload a")].map(link => link.href); console.log(links))()

Upvotes: 1

Related Questions