Creating a testing infrastructure with Pytest , Selenium Grid and Docker

Question

Based on this article, I successfully created the scalable selenium grid . Then I want to run my test suites [written in Python] using Pytest into that Grid.

I am new to Docker and trying to find the best way to migrate my testing procedures into the microservices architecture. Essentially I want to give the ability to the devs to setup a full testing infrastructure locally on their PCs.

So, I have 4 Dockerfiles and 1 docker-compose.yml .

The BASE dockerfile:

FROM ubuntu

ENV SEL_VERSION 2.44.0

# Update repos for Java
RUN apt-get update -qqy \
&& apt-get -qqy --no-install-recommends install \
software-properties-common \
&& rm -rf /var/lib/apt/lists/*
RUN add-apt-repository -y ppa:webupd8team/java

RUN echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections
RUN echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections

# Install Java7
RUN apt-get update -qqy \
&& apt-get -qqy --no-install-recommends install \
oracle-java7-installer \
&& rm -rf /var/lib/apt/lists/*

# Download Selenium Server
RUN wget http://selenium-release.storage.googleapis.com/${SEL_VERSION%.*}/selenium-server-standalone-${SEL_VERSION}.jar

The HUB dockerfile :

FROM org/grid:base

EXPOSE 4444

# Add and set permissions to the script that will launch the hub
ADD register-hub.sh /var/register-hub.sh
RUN chmod 755 /var/register-hub.sh

# start a shell and run the script
#WORKDIR /
CMD ["/bin/bash", "/var/register-hub.sh"]

The NODE dockerfile :

FROM org/grid:base

# set the FF version to use
ENV FIREFOX_MINOR 34.0.5

# Update and install what's needed
RUN apt-get update -qqy \
&& apt-get -qqy --no-install-recommends install \
firefox \
xvfb \
bzip2 \
&& rm -rf /var/lib/apt/lists/*

# setup FF
RUN [ -e /usr/bin/firefox ] && rm /usr/bin/firefox
ADD https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/${FIREFOX_MINOR}/linux-x86_64/en-US/firefox-${FIREFOX_MINOR}.tar.bz2 /tmp/
RUN apt-get install -q -y libdbus-glib-1-2
RUN tar -xvjf /tmp/firefox-${FIREFOX_MINOR}.tar.bz2 -C /opt/
RUN chmod -R +x /opt/firefox/
RUN ln -s /opt/firefox/firefox /usr/bin/firefox

# add and set permissions for the bash script to register the node
ADD register-node.sh /var/register-node.sh
RUN chmod 755 /var/register-node.sh

# start a shell and run the script
CMD ["/bin/bash", "/var/register-node.sh"]

And the PYTEST dockerfile :

# Starting from base image
FROM ubuntu

# Set the Github personal token [to clone the QA code]
#todo------secret------
ENV GH_TOKEN some_token_some_token_some_token_

# Install Python & pip
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -y python python-pip python-dev && pip install --upgrade pip

# Install GIT
RUN apt-get update -y && apt-get install git -y

# [in the / folder] Create the folder and add the whole project from the repo to the container
RUN git clone https://$GH_TOKEN:x-oauth-basic@github.com/org/org_QA.git /org_QA_folder

# Install dependencies via pip
WORKDIR /org_QA_folder
RUN pip install -r dependencies.txt

#
CMD /bin/bash

And the docker-compose file :

hub:
  image: org/grid:hub # image must already exist
  ports:
    - "4444:4444" # HOST:CONTAINER
  external_links: # link to a container created outside of this YAML file
    - pytest
  volumes_from:
    - pytest # must be created first

firefox:
  image: org/grid:nodeff # image must already exist
  links:
    - hub
  expose:
    - "5555" # grid console open to public

So...I cannot understand how should I run something like "py.test /org_QA_folder/testcase.py" and run on the grid [essentially the node].

I first run the pytest container with docker run -dit -v /org_QA_folder --name pytest schoox/grid:py and the other services with docker-compose up -d .

I tried multiple approaches:

1. Not use a Pytest container but clone the code in the hub. But in this case , HUB container runs the hub of the grid [one CMD per container directive] . [EDIT: I chose the microservices architecture as Docker suggests]

2. I leave the hub and node containers to run their specified services, and create a separate container [pytest] to send the tests from there. But the test wont run cause this container does not have xvfb.[EDIT: I just installed xvfb and worked]

3. I tried to use the pytest container as volume for the hub but remains the problem of running the tests from the hub container that already has a service running.[EDIT: For dev purposes I mount the code from the pytest container and use its console to test stuff on the other containers. After that I will use the pytest container to have the code separate]

Should I install all the packages ,that node container has, in the other containers? Ideally I want to use the pytest container console [to develop the infrastructure at least] to run the tests which will be sent to hub and run to many nodes.[EDIT: I didn't mingled with the hub and node, I just installed xvfb on pytest container]

How would you suggest to do that? Am I missing something ?

EDIT [steps towards solution]:

I first start the pytest container [docker run -dit -v /org/org_QA_folder --name pytest org/grid:py] and then I run docker-compose up -d to launch the grid

In the compose file I use external_links and volumes_from for dev purposes. I wont work from inside any container when the setup is finished.

I changed the Pytest dockerfile and added this:

# Install xvfb [THAT WAS NEEDED!!!!!]
RUN apt-get install -y xvfb

I run one of my tests ,from inside the pytest container, to my already running Grid[on an EC2 instance] as a proof of concept and worked fine. It remains to send it to the dockerized grid.

Answer 1

I finally made it to work like this:

From my code I use this:

class Fixtures_docker(unittest.TestCase):

    def setUp(self, hub_ip="http://HUB_CONTAINER_IP", hub_port=4444):
        print "--Fixtures_docker in config.py trying to run on--: %s:%s"%(hub_ip, hub_port)

        if platform.system() == 'Linux':
            from pyvirtualdisplay import Display

            self.display = Display(visible=0, size=(1024, 768))
            self.display.start()

        if browser == "Firefox": 
            self.wd = webdriver.Remote(
                command_executor=hub_ip + ':' + str(hub_port) + '/wd/hub',
                    desired_capabilities={
                        #FIREFOX
                        "browserName"   : "firefox",
                        # "maxInstances"  : 10,
                        "seleniumProtocol": "WebDriver",
                        "platform"      : "ANY",
                        "node"          : hub_port
                })

        # for debugging purposes from local machine
        elif browser == "Firefox" and os == "Windows":
            self.wd = webdriver.Remote(
                command_executor=hub_ip + ':' + str(hub_port) + '/wd/hub',
                    desired_capabilities={
                        # FIREFOX -> WINDOWS
                        # "firefox_binary":"C:\\firefox3\Firefox3.exe", # path to find another version of ff
                        "browserName"   : "firefox",
                        #"version"      : "3.0b3", # trying to run a specific ff version
                        "platform"      : "WINDOWS",
                        "node"          : hub_port,
                        "maxSession"    : 5
                })

        self.wd.implicitly_wait(10)

    def tearDown(self):
        self.wd.quit()
        if platform.system() == 'Linux':
            self.display.stop()

and in the docker-compose.yml:

hub:
  image: org/grid:hub # image must already exist
  ports:
    - "8080:4444" # HOST:CONTAINER
  external_links: # link to a container created outside of this YAML file
    - pytest
  volumes_from:
    - pytest # must be created first
  expose:
    - "4444" # intercontainer communication [other containers with selenium server]

firefox:
  image: org/grid:nodeff # image must already exist
  links:
    - hub
  expose:
    - "5555" # intercontainer communication [node registers to hub]

Answer 2

I think you're almost there.

I would add a pytest service to the docker-compose.yml, and remove the volumes_from and external_links from the hub image (so that it looks more like the example from the blog post you linked).

The pytest service will run the python code, but it should connect to hub to run selenium. I believe somewhere in the org_QA_folder there is configuration that tries to start up selenium locally. It needs to be reconfigured to use the hub server instead.