selenium.common.exceptions.WebDriverException: Message: chromedriver unexpectedly exited. Status code was: 255 When using Dockerfile

Question

I have a Python Webscraping app that works flawlessly on my localhost (MacOS, M1 Silicon).

I am trying to publish it into Azure using a Container.

The problem:

When I build my app using a Dockerfile - the containerised image produces an error:

selenium.common.exceptions.WebDriverException: Message: Service /root/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: 255

The Dockerfile:

Here is the Dockerfile that I've created:

FROM python:3.9-buster
# FROM --platform=linux/amd64  python:3.9

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

RUN apt-get update \
    && apt-get -y install gcc make \
    && rm -rf /var/lib/apt/lists/*s

# RUN apt-get update
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium
# RUN apt-get install -y chromium-browser

# install manually all the missing libraries
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils

# install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
# RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install

RUN dpkg -i google-chrome-stable_current_amd64.deb --fix-missing; apt-get -fy install


RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get -y update

RUN apt-get update && apt-get install -y wget bzip2 libxtst6 packagekit-gtk3-module libx11-xcb-dev libdbus-glib-1-2 libxt6 libpci-dev && rm -rf /var/lib/apt/lists/*

#download and install chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install

RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
RUN apt-get install -y chromium

RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | tee /etc/apt/sources.list.d/google-chrome.list
RUN apt update -y
RUN apt install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils
# RUN apt install -y google-chrome-stable


RUN apt-get install -yqq unzip
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/
RUN python3 --version
RUN pip3 --version
RUN pip install --no-cache-dir --upgrade pip


#install python dependencies
COPY requirements.txt requirements.txt
RUN pip install -r ./requirements.txt

#some envs
ENV APP_HOME /app
ENV PORT 5000

#set workspace
WORKDIR ${APP_HOME}

#copy local files
COPY . .

CMD exec gunicorn --bind :${PORT} --workers 1 --threads 8 main:app
# CMD exec gunicorn -b :$PORT main:app
# EXPOSE 8080
# CMD ["gunicorn", "--bind", "0.0.0.0:8080","--timeout", "90", "main:main"]
# CMD ["python3", "main.py"]
#build using:
# docker build -t python-webscraper .
# docker run --rm -p 3500:5000 python-webscraper
# docker run -p 3500:5000 python-webscraper

Calling ChromeDriver from Code

chromedriver_autoinstaller.install()  # Check if the current version of chromedriver exists
    # and if it doesn't exist, download it automatically,
    # then add chromedriver to path
    chrome_options = Options()
    chrome_options.add_argument("--no-sandbox")
    # chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--headless")
    # chrome_options.add_argument("window-size=1400,2100")
    chrome_options.add_argument('--disable-gpu')
    chrome_options.add_argument("--disable-setuid-sandbox")
    chrome_options.add_argument('--disable-dev-shm-usage')  # Not used

As you can see, I'm using the headless option and no sandbox options to pass into the driver.

How am I building the container

I appreciate Im using MacOs , M1 chip, so I've tried to build using the following 2 ways:

docker buildx build --platform linux/amd64 -t app2 .

or

docker build -t app2

Then I just run it using

 docker run -it  app2

Summary

1. Im creating a local containerised image of my app to test it before publishing it to Azure.
2. The Dockerfile creates the container for me.
3. When I test it (run the image) locally - the web scraping app fails when launching Chrome Driver.

I'm now really desperate and have tried a lot of searching and refactoring without any luck.

Answer 1

I had the same issue. The problem was in the docker virtual environment. Adding the "options.add_argument('--disable-dev-shm-usage')" argument helped me. Chrome must be installed.

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service


service = Service(executable_path=ChromeDriverManager().install())
options = Options()
options.add_argument('--disable-blink-features=AutomationControlled') ## to avoid getting detected
options.add_argument('headless')
options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(service=service, options=options)"