CODEWITHSUNDEEP
python-3.xweb-scrapingchromiumpyppeteer
Petr Matuska
Petr Matuska

Reputation: 573

Pyppeteer crushes after 20 seconds with pyppeteer.errors.NetworkError

During usage of pyppeteer for controlling the Chromium I have been receiving an error approximately after 20 seconds of work:

pyppeteer.errors.NetworkError: Protocol Error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.

As described here the issue is probably caused by implementation of python websockets>=7 package and by its usage within pyppeteer.

Upvotes: 1

Views: 1017

Answers (1)

Petr Matuska
Petr Matuska

Reputation: 573

There are 3 solutions to prevent disconnection from Chromium:

- Patching the code like described here (preferable):
Run the snippet before running any other Pyppeteer commands

def patch_pyppeteer():
    import pyppeteer.connection
    original_method = pyppeteer.connection.websockets.client.connect

    def new_method(*args, **kwargs):
        kwargs['ping_interval'] = None
        kwargs['ping_timeout'] = None
        return original_method(*args, **kwargs)

    pyppeteer.connection.websockets.client.connect = new_method

patch_pyppeteer()

- Change the trouble making library:
Downgrade websockets package to websockets-6.0 e.g via pip3 install websockets==6.0 --force-reinstall (in your virtual environment)

- Change the code base as described in this pull request, which will be hopefully merged soon.

Upvotes: 3

Related Questions