CODEWITHSUNDEEP
javascriptpuppeteer
Reid Rivenburgh
Reid Rivenburgh

Reputation: 403

Node/Puppeteer: trying to get all links using selector, getting attribute of results

I'm new to puppeteer, and I'm somewhat familiar with javascript. I'm writing a node-based program. I'm running into some weird behavior, and I assume I'm just doing something dumb. I load a google image search results page, and I'm trying to get all of the result links with the selector 'a.islib'. This selector works correctly in the browser/dev tools. In my program, I'm doing:

const links = await page.$$eval('a.islib', as => as.map(a => a.href));

The number of array entries is correct, but they're all the empty string. I switched to this just to see what's going on:

const links = await page.$$eval('a.islib', as => as.map(a => a));

The first entry is "__jsaction":{"click":"J9iaEb"}, which is one of the attributes of the first anchor element. Why is it not returning the full anchor element itself?!

Thanks in advance. (P.S. I seem to be using puppeteer version 1.20.)

ETA: This is the link I'm processing:

https://www.google.com/search?q=dogs&hl=en&tbm=isch&tbs=rimg:CVk7HmLev_17XImDPO8esV0arKADKdM7jqM7dCa_1-61MenSKaBTjYm-qrRUA5ZBN55gHxKypVIfrnXZb0_1rIufBzLXmFELBm6_1vytevXlQkwmJqMP1Necxlbne6zAfmCcFk1wEP9DLWWljV0qEgnPO8esV0arKBEjPzeqLvmlKSoSCQDKdM7jqM7dEcto8VTESawhKhIJCa_1-61MenSIRIe3dPxnC54YqEgmaBTjYm-qrRRGEiFoX72OrQSoSCUA5ZBN55gHxEc5qRFT4QSO7KhIJKypVIfrnXZYR_1RulX7C8TYQqEgn0_1rIufBzLXhEsefAUHeutqyoSCWFELBm6_1vytEcto8VTESawhKhIJevXlQkwmJqMRuHNtQBKeSY0qEgkP1NecxlbnexFS8G2tqKwyOioSCazAfmCcFk1wESYDiIqo6UMbKhIJEP9DLWWljV0RNcI2y-eGBPdhxHWgL7B9uK8&tbo=u&sa=X&ved=2ahUKEwjZ0-izmJXoAhVOqZ4KHckfBhwQrnZ6BAgBEBc

This is the first element I get with the a.islib selector (with most of the image data removed to keep it somewhat short). My code is extracting the jsaction attribute listed below.

<a class="wXeWr islib nfEiy mM5pbd" jsname="sTFXNd" jsaction="click:J9iaEb;" data-nav="1" tabindex="0" style="height: 159px;"><div class="bRMDJf islir" jsname="DeysSe" style="width: 283px; height: 159px; margin-right: -14px;" jsaction="mousedown:npT2md; touchstart:npT2md;"><img class="rg_i Q4LuWd tx8vtf" src="data:image/jpeg;base64,/9j/4AA..." data-deferred="1" jsname="Q4LuWd" alt="Image result for dogs" data-iml="639.485000167042" data-atf="true"></div><div class="c7cjWc"></div><div class="h312td " jsname="bOERI"><span class="gRqDMe "><div class="lMSpef"><div class="O1vY7" aria-label="Click for video information"><span class="I1wio LyzHgf"><svg viewBox="0 0 24 24" focusable="false" class="MbCJkd" height="12" width="12"><path d="M0 0h24v24H0z" fill="none"></path><path d="M8 5v14l11-7z"></path></svg></span><span class=" RtIwE">13:12</span></div></div></span></div><div class="PiLIec" jsaction="click: gFs2Re"></div></a>

ETA2: There is no href in this. Still, it seems like my second await line above should return the entire element. How would I go about getting an array of them given the selector 'a.islib', from which I can then either click it or extract attributes like href? TIA.

Upvotes: 3

Views: 1139

Answers (1)

Reid Rivenburgh
Reid Rivenburgh

Reputation: 403

Here's what I've done, for anyone's edification. This will get the elements, click on them (which will update the href in this case), and then get the href. Seems to work for me, and hopefully the proper way to do this.

 const links = await page.$$('a.islib', as => as.map(a => a));
 for (const link of links) {
    await link.click();
    const updatedHref = await page.evaluate(a => a.href, link);
 }

Upvotes: 1

Related Questions