Puppeteer: How to click element so it opens in new tab?

Question

I have got a list of 25 elements that are clickable. I need to open each one of them in a new tab, scrape the new page opened in a new tab, then close it. Then go to the next element and do the same for each element in the list.

however, I am having problems opening the links in a new tab by clicking on them. I managed to open then with page.goto('link") but I want to make it more humanized and instead of pasting the link into the new tab I want to be opened by clicking.

let accountsClickElements = await page.$$eval(".result-lockup__name a", el => el.map(x => x.getAttribute("id")));
                                                                                        
 for (let i = 0; i<25; i++) {
     await autoScroll(page);
     await page.waitFor(3000);
     let id = companies[i];
     await page.focus("#"+accountsClickElements[0]);
     accountsClickElements = await page.$$eval(".result-lockup__name a", el => el.map(x => x.getAttribute("id")));
     await page.waitFor(3000);
     await page.focus("#"+accountsClickElements[i]);                             
     await page.click("#"+accountsClickElements[i]);
     await page.waitFor(2000);
     console.log("#"+companies[i].linid);
     await page.goBack(); 
  }                              `

This is a code that opens the links in the same tab, but after a while, it doesn't take all 25 elements and since id is changing every time I open the page I get an error.

I have changed the code like this so instead of clicking of the element, it is clicking on the href attribute. the target _blank attribute is there, but it still opening on the same tab. Can you point why?

await page.$$eval('.result-lockup__name a', el => el.map(x => x.setAttribute("target", "_blank")));
let accountsClickElements = await page.$$eval(".result-lockup__name a", el => el.map(x => x.getAttribute("href")));                                                                                                                  
for (let i = 0; i<25; i++) {
   await page.waitFor(2000);
   await autoScroll(page);
   await page.waitFor(2000);    
   await page.click('a[href="'+accountsClickElements[i]+'"]');
}

Answer 1

On Windows or Linux, hold down the Ctrl key before you click:

    // Ctrl+click to open in new tab
    await page.keyboard.down('Control');
    await page.click('a[href="'+accountsClickElements[i]+'"]')
    await page.keyboard.up('Control');

Then find the tab that was opened. One way to do that is with await browser.pages() and filtering out the current page. Note that when you start Puppeteer it already has one tab open, so if the first thing you did was const page = await browser.newPage(); you probably have two tabs and need to .close() the first one.

Here's a more complete code sample:

import puppeteer from 'puppeteer';

(async () => {
    const browser = await puppeteer.launch();
    // When the browser launches, it should have one about:blank tab open.
    const page = (await browser.pages())[0];

    // A safer way to do the above is to open a new tab and then close all
    // tabs that aren't the tab you just opened.
    // const page = await browser.newPage();
    // // Close any tabs that aren't the one we created on the line above
    // for (const p of (await browser.pages())) {
    //     if (p !== page) {
    //         await p.close()
    //     }
    // }

    await page.goto('https://example.com/');

    // Ctrl-click the "More information..." link to open it in a new tab
    await page.keyboard.down('Control');
    await page.click('a');
    await page.keyboard.up('Control');

    // Wait a second for the tab to open
    await page.waitForTimeout(1000);

    // Print the URLs of all currently open tabs
    console.error((await browser.pages()).map(p => p.url()));

    if ((await browser.pages()).length !== 2) {
        throw "unexpected number of tabs";
    }
    const otherPage = (await browser.pages())[1];
    // Do something with the other tab
    // ...
    // Then close it
    await otherPage.close();

    await browser.close();
})();

On macOS this won't work. The shortcut is Command+click so you would have to do await page.keyboard.down('Meta') instead of 'Control', but that doesn't work because on macOS keyboard shortcuts that use the Command key are usually handled by the operating system, not the browser (things like ⌘-A, which normally selects all text, don't work either). You can try

• middle clicking, as suggested by @nightowl
• Shift-clicking, which opens the link in a new window. Use browser.browserContexts() to find the new window
• running your code under Linux using a virtual machine or Docker container

Answer 2

import puppeteer from 'puppeteer'

(async () => {
    try {
        // Goal function to wait new page and get instance
        async function getNewBrowserTab(browser) {
            let resultPromise

            async function onTargetcreatedHandler(target) {
                if (target.type() === 'page') {
                    const newPage = await target.page()
                    const newPagePromise = new Promise(y =>
                        newPage.once('domcontentloaded', () => y(newPage))
                    )

                    const isPageLoaded = await newPage.evaluate(
                        () => document.readyState
                    )

                    browser.off('targetcreated', onTargetcreatedHandler) // unsubscribing

                    return isPageLoaded.match('complete|interactive')
                        ? resultPromise(newPage)
                        : resultPromise(newPagePromise)
                }
            }

            return new Promise(resolve => {
                resultPromise = resolve
                browser.on('targetcreated', onTargetcreatedHandler)
            })
        }

        // Using
        const browser = await puppeteer.launch({ headless: false })
        const page = await browser.newPage()

        await page.goto('https://www.google.com/')

        // Click on link with middle button to open in new browser tab
        await page.click('a[href]', { button : 'middle' })

        // Wait for new tab and return a page instance
        const newPage = await getNewBrowserTab(browser)

        // Switch to new tab
        await newPage.bringToFront()

        // Wait a bit to see the page
        await new Promise(resolve => setTimeout(resolve, 1000))

        await newPage.close()
        await page.close()
        await browser.close()
    } catch (e) {
        console.error(e)
    }
})()

Answer based on puppeteer github

Answer 3

Middle click using page.click:

let options = {button : 'middle'};
await page.click('a[href="'+accountsClickElements[i]+'"]', options)

It might take some time for the new tab to open, which you can wait for with await page.waitForTimeout(1000).

Full code:

let accountsClickElements = await page.$$eval('.result-lockup__name a', el => el.map(x => x.getAttribute('href')));

browser.on('targetcreated', function(){                                            
    console.log(accountsClickElements[i]) 
})   

let options = {
    button : 'middle'
};
for (let i = 0; i<25; i++) {
    await page.waitForTimeout(2000);
    await page.focus('a[href="'+accountsClickElements[i]+'"]')                                                    
    await page.click('a[href="'+accountsClickElements[i]+'"]', options)
    const [tab1, tab2, tab3] = await browser.pages();
    await page.waitForTimeout(2000);
    await tab3.bringToFront();                                                                               
    await ListenCompanyPageNewTab(tab3); 
    await tab2.bringToFront();                                                            
} 

Answer 4

add target="_blank" to the elements you would like to click:

await page.$$eval('.result-lockup__name a', el => el.map(x => x.setAttribute("target", "_blank")));

Answer 5

You can create a page first on the browser instance and then goto that page.

const page = await browser.newPage();

await page.goto('https://url.com');