CODEWITHSUNDEEP
pythonseleniumselenium-chromedriverremote-debugginggoogle-chrome-headless
Norhther
Norhther

Reputation: 500

Selenium Chrome Webdriver not working in headless mode with profile

So, this is the code I'm having troubles with:

def scrap():
        options = webdriver.ChromeOptions();
        options.add_argument('headless');
        options.add_argument('--profile-directory=Profile 1')
        options.add_argument("--user-data-dir=C:/Users/omarl/AppData/Local/Google/Chrome/User Data/")
        options.add_argument("--remote-debugging-port=45447")
    
        options.add_argument("--disable-gpu") 
        browser = webdriver.Chrome(executable_path=r"C:\Users\omarl\OneDrive\Escritorio\chromedriver.exe", options=options)
        
        scrapURL = "https://es.wallapop.com/search?distance=30000&keywords=leggins&latitude=41.38804&longitude=2.17001&filters_source=quick_filters"
        browser.get(scrapURL)
        #...

And the error:

WebDriverException: unknown error: unable to discover open pages

I don't have any instances of chrome when I execute the script, and when I'm using it without the headless option it works fine. Any idea why this is happening? Please, note that I'm using the --remote-debuggin-port provided in similar questions.

I'm using ChromeDriver 86.0.4240.22

Upvotes: 5

Views: 9219

Answers (2)

undetected Selenium
undetected Selenium

Reputation: 193058

To invoke a Chrome Profile in Headless mode you can use the --user-data-dir argument only and you can safely remove --profile-directory argument as follows:

  • Code Block:

    from selenium import webdriver

options = webdriver.ChromeOptions() 
options.add_argument('--headless')
options.add_argument('--window-size=1920,1080')
# options.add_argument('--profile-directory=Profile 1')
options.add_argument(r"--user-data-dir=C:\Users\Soma Bhattacharjee\AppData\Local\Google\Chrome\User Data\Default")
options.add_argument("--remote-debugging-port=9222")
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.google.com/')
print("Chrome Headless launched")

  • Console Output:

    DevTools listening on ws://127.0.0.1:9222/devtools/browser/93c67c41-e125-4d12-abc0-fcf0f07a62f4
Chrome Headless launched

References

You can find a couple of relevant detailed discussions in:

Additional Considerations

Ensure that:

  • Selenium is upgraded to current released Version 3.141.0.
  • ChromeDriver is updated to current ChromeDriver v86.0 level.
  • Chrome is updated to current Chrome Version 86.0 level. (as per ChromeDriver v86.0 release notes).
  • Execute your @Test as non-root user.
  • Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.

tl; dr

ChromeDriver remote debug port reservation race conditions

Upvotes: 2

user7788539
user7788539

Reputation: 89

Have you tried using arg --no-sandbox? A lot of people on Chrome Driver Error using Selenium: Unable to Discover Open Pages have had success with that argument.

Upvotes: 0

Related Questions