CODEWITHSUNDEEP
javascriptpuppeteergoogle-chrome-headless
Fahad
Fahad

Reputation: 3

Reloading page in puppeteer if response code is not valid

So I know I can check the response code after a page.goto() call using the response.status() function but my program is built to scrape and do a bunch of actions on a website. Now some websites under load or randomly return a 500 error or 503 error instead of serving up the webpage.

So what I want to do is, for every navigation request, if the response code returns a 500 or 503 error, I want to reload the page. I have been taking a look at setRequestInterception but that's fired before a request is made. setResponseInterception doesn't exist yet (but I see it as a potential feature in github). It would be a peace of cake with setResponseInterception:

  1. Check response code
  2. If 500 or 503, reload page

I am wondering if I can do anything like this right now using setRequestInterception. Or I may have to individually monitor each navigation call and check if it returns a valid code before proceeding.

Upvotes: 0

Views: 2619

Answers (1)

hretic
hretic

Reputation: 1095

you didn't provide any code sample so i dont know who your code structure is but here is 1 way to do this 

async function init_puppeteer( test  ) {


    var browser = await puppeteer.launch({headless: false    ,  args: ['--no-sandbox', '--disable-setuid-sandbox' , ]});

    let success = false ;
    while(!success)
    {
       success = await open_page( browser , link );
    }
    browser.close();
}


async function open_page( browser , link ){

    try 
    {
        const page = await browser.newPage();
        await page.goto( link ).catch(function (error) {  throw new Error('TimeoutBrows');  });

        // also you can check status code and throw error if its 500 or 503

        return true ; 
    }
    catch(e){
        return false; 
    }
}

Upvotes: 1

Related Questions