CODEWITHSUNDEEP
pythonseleniumxvfbpyvirtualdisplay
绿飕飕
绿飕飕

Reputation: 23

How dose pyvirtualdisplay work on Xvfb or how many Xvfb could be open by pyvirtualdisplay?

  1. I had used the following code to open saveral Xvfb for test and got some problems as:
    A. Some process of Xvfb's status(following the code) were Z or SL,sometime were Z+ or SL+ , but python process was going on normally What dose it mean?
    B. The code run normally on the begging, but got exception(following the status) on the end.

Code 

#!/usr/bin/env python3
# encoding: utf-8
import os
import time
import random
import multiprocessing
from pyvirtualdisplay import Display
from selenium import webdriver

def main():
    display = Display(visible=0, size=(800, 600))
    display.start()
    print(os.getpid())
    browser = webdriver.Firefox()
    #time.sleep(20)
    for j in range(30):
        browser.get('http://www.google.com')
        print(browser.title)
        time.sleep(1) # !!!!!! this is sleep time
    browser.quit()
    display.stop()

if __name__ == '__main__':
    tl = []
    for i in range(10):
        tl.append(multiprocessing.Process(target=main))
    start = time.time()
    for j in tl:
        j.start()
    for j in tl:
        j.join()
    print("end {}".format((time.time() - start)))

STATUS

root     18503  0.0  0.0      0     0 ?        Z    13:40   0:00 [Xvfb] <defunct>
root     18504  0.1  0.7 233684 57364 ?        Sl   13:40   0:22 Xvfb -br -nolisten tcp -screen 0 800x800x24 :1049
root     18506  0.0  0.0      0     0 ?        Z    13:40   0:00 [Xvfb] <defunct>
root     18508  0.0  0.0      0     0 ?        Z    13:40   0:00 [Xvfb] <defunct>
root     18509  0.0  0.0      0     0 ?        Z    13:40   0:00 [Xvfb] <defunct>
root     18514  0.0  0.5 221140 44736 ?        Sl   13:40   0:11 Xvfb -br -nolisten tcp -screen 0 800x800x24 :1052
root     18515  0.0  0.0      0     0 ?        Z    13:40   0:00 [Xvfb] <defunct>
root     18516  0.0  0.0      0     0 ?        Z    13:40   0:00 [Xvfb] <defunct>
root     18517  0.0  0.0      0     0 ?        Z    13:40   0:00 [Xvfb] <defunct>
root     18520  0.0  0.4 212832 36468 ?        Sl   13:40   0:04 Xvfb -br -nolisten tcp -screen 0 800x800x24 :1055

EXCEPTION

Process Process-10:
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/root/PycharmProjects/MyShiYanLou/auto_/test_xvfb.py", line 27, in main
    browser.quit()
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/firefox/webdriver.py", line 183, in quit
    RemoteWebDriver.quit(self)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 592, in quit
    self.execute(Command.QUIT)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 297, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette

Upvotes: 2

Views: 2254

Answers (1)

puchal
puchal

Reputation: 2389

The exception you've got is not caused by the fact that pyvirtualdisplay can open certain amount of Xvfb, but by the fact that selenium will be opened in the last virtual display you've opened.

Imagine the situation where there are two processes A and B

  1. A opens Xvfb
  2. B opens Xvfb
  3. A opens webdriver in virtual display opened by B (the last one that was opened)
  4. Bcloses Xvfb
  5. A tries to get web page but it fails since Xvfb where webdriver was opened is already closed

The best way to avoid that error is to open one Xvfb before running multiple processes:

Code

#!/usr/bin/env python3
# encoding: utf-8
import os
import time
import multiprocessing
from pyvirtualdisplay import Display
from selenium import webdriver

def main():
    print(os.getpid())
    browser = webdriver.Firefox()
    for j in range(30):
        browser.get('http://www.google.com')
        print(browser.title)
        time.sleep(1)  # !!!!!! this is sleep time
    browser.quit()


if __name__ == '__main__':
    display = Display(visible=0, size=(800, 600))
    display.start()

    tl = []
    for i in range(10):
        tl.append(multiprocessing.Process(target=main))
    start = time.time()
    for j in tl:
        j.start()
    for j in tl:
        j.join()

    display.stop()

    print("end {}".format((time.time() - start)))

Upvotes: 2

Related Questions