Running a Selenium(Python) based app in Docker

Question

I'm trying to dockerize and run the web scrapper developed using the selenium library in python. I used Windows 10 for development. It ran well there. While running the same script as a docker image, I'm getting multiple issues. This is how I connect the driver in windows.

    driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))

I didn't use options as I don't have any use cases. As I got root user error while running in docker I added the option and ran the code as below.

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
driver = webdriver.Chrome(options = chrome_options, service=Service(ChromeDriverManager().install()))

Still, it didn't start. So I configured it by hardcoding the driver path.

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
driver = webdriver.Chrome(executable_path=driverPath,options=option)

Even then it didn't get started as the display was not configured. So configured the headless argument and ran, but in the end, I got the below error.

**

Tkinter.TclError: no display name and no $DISPLAY environment variable

**

So I tried to start the display by the below code.

if platform.system() == 'Linux':
        from pyvirtualdisplay import Display
        display = Display(visible=0, size=(800, 800))  
        display.start()

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
driver = webdriver.Chrome(executable_path=driverPath,options=option)

But it is not running, it is frozen and not creating the driver session.

This is my Dockerfile

FROM python
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \ 
    && echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list
RUN apt-get update && apt-get -y install google-chrome-stable
RUN apt-get install -yqq unzip
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/
RUN apt-get install xvfb mesa-utils -y \
        && apt install freeglut3-dev -y
ENV DISPLAY=:99
RUN mkdir -p /app/drivers
ADD requirements.txt /app
ADD sample.py /app
COPY run.sh /app
COPY drivers /app/drivers
COPY csv /app/csv
WORKDIR /app
RUN pip3 install -r requirements.txt
CMD ./run.sh

run.sh

#!/bin/sh

#Xvfb :99 -screen 0 640x480x8 -nolisten tcp &
python3 ./sample.py 

requirements.txt

selenium==4.3.0
webdriver-manager==3.8.2
chromedriver-py==103.0.5060.53
pyvirtualdisplay==3.0

What are the mistakes I made in the code? And how to run the selenium python app with display in docker? Thank you.

Answer 1

Seems the display can't be enabled in the python jar. So I have created the python image from the ubuntu image as said in this site. There I have installed the python and the other dependencies required for my application. And now I'm able run the application without any issues.

FROM ubuntu
#Enabling noninteractive environment and setting Timezone to install python3-tk without any interruption
# python
RUN export TZ=Asia/Kolkata
RUN apt-get update
RUN apt-get install -y python3 python3-setuptools python3-pip python3-tk

ENV DEBIAN_FRONTEND noninteractive 
# Essential tools and xvfb
RUN apt-get update && apt-get install -y \
    software-properties-common \
    unzip \
    curl \
    xvfb 
 
# Chrome browser to run the tests
RUN curl https://dl-ssl.google.com/linux/linux_signing_key.pub -o /tmp/google.pub \
    && cat /tmp/google.pub | apt-key add -; rm /tmp/google.pub \
    && echo 'deb http://dl.google.com/linux/chrome/deb/ stable main' > /etc/apt/sources.list.d/google.list \
    && mkdir -p /usr/share/desktop-directories \
    && apt-get -y update && apt-get install -y google-chrome-stable
# Disable the SUID sandbox so that chrome can launch without being in a privileged container
RUN dpkg-divert --add --rename --divert /opt/google/chrome/google-chrome.real /opt/google/chrome/google-chrome \
    && echo "#!/bin/bash\nexec /opt/google/chrome/google-chrome.real --no-sandbox --disable-setuid-sandbox \"\$@\"" > /opt/google/chrome/google-chrome \
    && chmod 755 /opt/google/chrome/google-chrome
 
# Chrome Driver
RUN mkdir -p /opt/selenium \
    && curl http://chromedriver.storage.googleapis.com/2.45/chromedriver_linux64.zip -o /opt/selenium/chromedriver_linux64.zip \
    && cd /opt/selenium; unzip /opt/selenium/chromedriver_linux64.zip; rm -rf chromedriver_linux64.zip; ln -fs /opt/selenium/chromedriver /usr/local/bin/chromedriver;

# display
RUN export DISPLAY=:20
RUN Xvfb :20 -screen 0 1366x768x16 &
RUN mkdir -p /app
ADD requirements.txt /app
ADD app.py /app
WORKDIR /app
RUN pip3 install -r requirements.txt
CMD ./run.sh