Puppeteer/Chromium Failing on Ubuntu Server

Question

I've written an application that runs on my Ubuntu 18.04 x64 server on Digital Ocean. I'm keeping the application running with PM2.

The code runs successfully for the first several iterations (it's a web scraper that runs every 15 minutes) but after a few hours the app breaks down. Here are the error logs (each starts with date):

info @ 03-20-2019 19:30:15 –– No updates –– 2019-20-03 // The is a successful log...
info @ 03-20-2019 19:30:15 –– No updates –– 2019-20-03 // The is a successful log...
info @ 03-20-2019 19:30:15 –– No updates –– 2019-20-03 // The is a successful log...
info @ 03-20-2019 19:30:15 –– No updates –– 2019-20-03 // The is a successful log...
info @ 03-20-2019 19:30:15 –– No updates –– 2019-20-03 // The is a successful log...
info @ 03-20-2019 19:30:15 –– No updates –– 2019-20-03 // The is a successful log...
debug @ 03-20-2019 19:45:44 –– Navigation Timeout Exceeded: 30000ms exceeded
debug @ 03-20-2019 20:15:20 –– Failed to launch chrome!
[0320/201518.129756:FATAL:zygote_host_impl_linux.cc(170)] Check failed: process.IsValid(). Failed to launch zygote process
#0 0x55e0a9ef8f29 base::debug::CollectStackTrace()
#1 0x55e0a9e5e593 base::debug::StackTrace::StackTrace()
#2 0x55e0a9e72d1e logging::LogMessage::~LogMessage()
#3 0x55e0ab4dbd79 service_manager::ZygoteHostImpl::LaunchZygote()
#4 0x55e0a9ab4950 content::(anonymous namespace)::LaunchZygoteHelper()
#5 0x55e0ab4db128 service_manager::ZygoteCommunication::Init()
#6 0x55e0ab4e1353 service_manager::CreateGenericZygote()
#7 0x55e0a9ab41fb content::ContentMainRunnerImpl::Initialize()
#8 0x55e0a9ae6fca service_manager::Main()
#9 0x55e0a9ab2791 content::ContentMain()
#10 0x55e0ae14e178 headless::(anonymous namespace)::RunContentMain()
#11 0x55e0ae14e205 headless::HeadlessBrowserMain()
#12 0x55e0a9ae5ca3 headless::HeadlessShellMain()
#13 0x55e0a7a1d1ac ChromeMain
#14 0x7f59a1e33b97 __libc_start_main
#15 0x55e0a7a1d02a _start

Received signal 6
#0 0x55e0a9ef8f29 base::debug::CollectStackTrace()
#1 0x55e0a9e5e593 base::debug::StackTrace::StackTrace()
#2 0x55e0a9ef8ab1 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#3 0x7f59a80aa890 <unknown>
#4 0x7f59a1e50e97 gsignal
#5 0x7f59a1e52801 abort
#6 0x55e0a9ef78e5 base::debug::BreakDebugger()
#7 0x55e0a9e72f61 logging::LogMessage::~LogMessage()
#8 0x55e0ab4dbd79 service_manager::ZygoteHostImpl::LaunchZygote()
#9 0x55e0a9ab4950 content::(anonymous namespace)::LaunchZygoteHelper()
#10 0x55e0ab4db128 service_manager::ZygoteCommunication::Init()
#11 0x55e0ab4e1353 service_manager::CreateGenericZygote()
#12 0x55e0a9ab41fb content::ContentMainRunnerImpl::Initialize()
#13 0x55e0a9ae6fca service_manager::Main()
#14 0x55e0a9ab2791 content::ContentMain()
#15 0x55e0ae14e178 headless::(anonymous namespace)::RunContentMain()
#16 0x55e0ae14e205 headless::HeadlessBrowserMain()
#17 0x55e0a9ae5ca3 headless::HeadlessShellMain()
#18 0x55e0a7a1d1ac ChromeMain
#19 0x7f59a1e33b97 __libc_start_main
#20 0x55e0a7a1d02a _start
  r8: 0000000000000000  r9: 00007ffc5ad954f0 r10: 0000000000000008 r11: 0000000000000246
 r12: 00007ffc5ad95798 r13: 000000000000007a r14: 00007ffc5ad96200 r15: 00007ffc5ad961f8
  di: 0000000000000002  si: 00007ffc5ad954f0  bp: 00007ffc5ad95740  bx: 000036f0ed7d7000
  dx: 0000000000000000  ax: 0000000000000000  cx: 00007f59a1e50e97  sp: 00007ffc5ad954f0
  ip: 00007f59a1e50e97 efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.


TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

    debug @ 03-20-2019 20:30:34 –– Timed out after 30000 ms while trying to connect to Chrome! The only Chrome revision guaranteed to work is r637110
    debug @ 03-20-2019 20:45:31 –– Timed out after 30000 ms while trying to connect to Chrome! The only Chrome revision guaranteed to work is r637110
    debug @ 03-20-2019 21:00:30 –– Timed out after 30000 ms while trying to connect to Chrome! The only Chrome revision guaranteed to work is r637110

debug @ 03-20-2019 21:15:05 –– Failed to launch chrome!
[0320/211503.503077:ERROR:platform_thread_posix.cc(129)] pthread_create: Resource temporarily unavailable (11)
[0320/211503.899485:FATAL:simple_thread.cc(56)] Check failed: success. 
#0 0x55e159dbef29 base::debug::CollectStackTrace()
#1 0x55e159d24593 base::debug::StackTrace::StackTrace()
#2 0x55e159d38d1e logging::LogMessage::~LogMessage()
#3 0x55e159d959ff base::SimpleThread::Start()
#4 0x55e15876edd2 content::SandboxHostLinux::Init()
#5 0x55e15997a125 content::ContentMainRunnerImpl::Initialize()
#6 0x55e1599acfca service_manager::Main()
#7 0x55e159978791 content::ContentMain()
#8 0x55e15e014178 headless::(anonymous namespace)::RunContentMain()
#9 0x55e15e014205 headless::HeadlessBrowserMain()
#10 0x55e1599abca3 headless::HeadlessShellMain()
#11 0x55e1578e31ac ChromeMain
#12 0x7f4000b8ab97 __libc_start_main
#13 0x55e1578e302a _start

Received signal 6
#0 0x55e159dbef29 base::debug::CollectStackTrace()
#1 0x55e159d24593 base::debug::StackTrace::StackTrace()
#2 0x55e159dbeab1 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#3 0x7f4006e01890 <unknown>
#4 0x7f4000ba7e97 gsignal
#5 0x7f4000ba9801 abort
#6 0x55e159dbd8e5 base::debug::BreakDebugger()
#7 0x55e159d38f61 logging::LogMessage::~LogMessage()
#8 0x55e159d959ff base::SimpleThread::Start()
#9 0x55e15876edd2 content::SandboxHostLinux::Init()
#10 0x55e15997a125 content::ContentMainRunnerImpl::Initialize()
#11 0x55e1599acfca service_manager::Main()
#12 0x55e159978791 content::ContentMain()
#13 0x55e15e014178 headless::(anonymous namespace)::RunContentMain()
#14 0x55e15e014205 headless::HeadlessBrowserMain()
#15 0x55e1599abca3 headless::HeadlessShellMain()
#16 0x55e1578e31ac ChromeMain
#17 0x7f4000b8ab97 __libc_start_main
#18 0x55e1578e302a _start
  r8: 0000000000000000  r9: 00007ffc5b48ace0 r10: 0000000000000008 r11: 0000000000000246
 r12: 00007ffc5b48af88 r13: 0000000000000047 r14: 00007ffc5b48b8e8 r15: 00007ffc5b48b8e0
  di: 0000000000000002  si: 00007ffc5b48ace0  bp: 00007ffc5b48af30  bx: 00000233a2484b00
  dx: 0000000000000000  ax: 0000000000000000  cx: 00007f4000ba7e97  sp: 00007ffc5b48ace0
  ip: 00007f4000ba7e97 efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.


TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

The puppeteer part of my program looks like this:

const fetchContracts = async (url) => {

    const browser = await pupeteer.launch({ headless: true, args: ['--no-sandbox', '--disable-setuid-sandbox']});
    const page = await browser.newPage(); // Create new instance of puppet
    const pendingXHR = new PendingXHR(page);

    await page.goto(url, { waitUntil: 'networkidle2' }); // Ensure no network requests are happening (in last 500ms).
    await Promise.all([
        page.click("#agree_statement"),
        page.waitForNavigation()
    ]);

    await page.click(".form-check-input");

    await Promise.all([
        page.click(".btn-primary"),
        page.waitForNavigation()
    ]);    

    await pendingXHR.waitForAllXhrFinished();
    await page.click('.sorting:nth-child(5)');
    await pendingXHR.waitForAllXhrFinished();
    await page.click('.sorting_asc');
    await pendingXHR.waitForAllXhrFinished();

    let html = await page.content();
    await page.close();
    return html;
}

Does anyone know what I am doing wrong? I have very little experience with stack traces and error logging. Thank you.

Answer 1

I forgot to quit out of the browser! I was quitting out of the page, but leaving the browser hanging, which was causing all sorts of memory problems on my server. Here's the PIDs on my server to prove what was going wrong.

After adding in await browser.close(); my code is working again!