CODEWITHSUNDEEP
pythonseleniumweb-scrapingweb-crawler
Lululu
Lululu

Reputation: 745

Docker Selenium Chromedriver: Unfortunately, automated access to this page was denied

I am using selenium chromedriver in my python project.

The application is running under Docker.

When I try to access http://mobile.de website I got rejected stating:

Unfortunately, automated access to this page was denied.

Here is my initialization code:

    CHROME_DRIVER_PATH = os.path.abspath('assets/chromedriver')
    chrome_options = ChromeOptions()
    chrome_options.binary_location = "/usr/bin/google-chrome"
    chrome_options.add_argument('--headless')
    chrome_options.add_argument('--no-sandbox')
    self.web_driver_chrome = webdriver.Chrome(executable_path=CHROME_DRIVER_PATH, options=chrome_options)

And here is my send request code:

def get_page_content(self, url):
    url = "https://www.mobile.de/"
    self.web_driver_chrome.get(url)
    print(self.web_driver_chrome.page_source)
    return self.web_driver_chrome.page_source

Is there any way I can pass this "automated access check"?

Upvotes: 1

Views: 424

Answers (1)

ewwink
ewwink

Reputation: 19154

when using --headless it append HeadlessChrome to the user-agent 

Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/71.0.3578.98 Safari/537.36

The solution is adding argument to set normal user-agent

user_agent = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'
chrome_options.add_argument('user-agent=' + user_agent)

Upvotes: 2

Related Questions