CODEWITHSUNDEEP
pythongitlab
Alberto Castro
Alberto Castro

Reputation: 13

Problem using MPI in a python program when testing it with a gitlab-ci.yml pipeline

I have a python code hosted in gitlab. Recently, I have added the use of to handle parallel runs with MPI. I have a testsuite that is launched by the gitlab CI system. However, it fails as soon as it tries to install the mpi4py package.

I am using the python 3.7 image. The .gitlab-ci.yml file is:

image: python:3.7

test:
  stage: test
  script:
    - pip install -r requirements.txt
    - python setup.py build
    - python setup.py install
    - pytest --junitxml=report.xml -v tests/short.py
  artifacts:
    when: always
    paths:
      - report.xml
    reports:
      junit: report.xml

The failure happens when, upon installing the mpi4py package as requested in the requirements.txt file, the build process fails with the message:

 gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.7m -c _configtest.c -o _configtest.o
      _configtest.c:2:10: fatal error: mpi.h: No such file or directory
          2 | #include <mpi.h>
            |          ^~~~~~~
      compilation terminated.
      failure.
      removing: _configtest.c _configtest.o
      error: Cannot compile MPI programs. Check your configuration!!!
      [end of output]

I suppose that the mpi C library is missing? How should I install it?

Upvotes: 1

Views: 168

Answers (1)

WArnold
WArnold

Reputation: 445

Maybe you want to have a look at how to install libmpich-dev in general (https://mpi4py.readthedocs.io/en/latest/install.html#linux).

gcc should be present in your container already.

in the .gitlab-ci.yml add these two lines:

EDIT If you have trouble finding the package run the following in you container

apt update

apt-get install -y libmpich-dev

you script should look like:

image: python:3.7

test:
  stage: test
  script:
    - apt update
    - apt-get install -y libmpich-dev
    - pip install -r requirements.txt
    - python setup.py build
    - python setup.py install
    - pytest --junitxml=report.xml -v tests/short.py
  artifacts:
    when: always
    paths:
      - report.xml
    reports:
      junit: report.xml

Upvotes: 1

Related Questions