puppeteer : cant log in and loop through urls

Question

Hi Guys I want to log in a website and once authenticated want to loop through a given set of URLS and scrape data. What I intend to do can be described by this example,however I get Unhandled promise rejection.

const puppeteer = require("puppeteer");

list = [
	"https://www.facebook.com/",
	"https://www.google.com/",
	"https://www.zocdoc.com/"
];

const getTitle = async (p, url) => {
    try{
        await p.goto(url);
        const title = await p.title();
        console.log(title);
    }
    catch(e) {
        console.log(e)
    }

    return title
};

(async () => {
	const browser = await puppeteer.launch();
    const page = await browser.newPage();
    console.log(this)
    for (var url of list) {
        getTitle(page, url)
    }
	await browser.close();
})();

Answer 1

There are multiple issues in this example.

1. You should await the call to function getTitle, you re awaiting inside the function but you have to await the call to the function too.

2. You should surround getTitle with a try and catch block and check inside the function if theres a title to return (ex. the title for google is null)

   const puppeteer = require("puppeteer");
   
   list = [
       "https://www.facebook.com/",
       "https://www.google.com/",
       "https://www.zocdoc.com/"
   ];
   
   const getTitle = async (p, url) => {
       try{
           await p.goto(url);
           const title = await p.title();
           if(title){
               return title
           }
       }
       catch(e) {
           throw(e)
           console.log(e)
       }
   };
   
   (async () => {
       const browser = await puppeteer.launch();
       const page = await browser.newPage();
       console.log(this)
       for (var url of list) {
           try{
               console.log(await  getTitle(page, url))
           }
           catch(e ){
               console.log('No title')
           }
       }
       await browser.close();
   })();