CODEWITHSUNDEEP
rpmpython-venv
mikew
mikew

Reputation: 924

Deploying python application internally using venv and rpm

I'm looking for others that have experience with python venv and deployment of applications via rpm. My goals are to:

  1. Use pip for python dependency
  2. Use venv to keep application environments/deps separate
  3. Use rpm for deployment (required by our companies internal audit etc).

I have a build server (jenkins slave) for each architecture (read: distro) that we deploy on. My original (and only) plan on the jenkins slave(build server) via a jenkins job was to:

  1. Create a venv
  2. Activate venv
  3. python setup.py build/install (within rpm spec)
  4. archive rpm as artifact
  5. Rejoice

I never got to step 2 or 3 so I do not know the dragons there, however the main issue comes with "Create a venv" step. Since venvs are not "relocatable" and RPM uses a RPM_BUILD_ROOT, which is a self contained filesystem in a tmpdir that we package from, I can't install the venv into the rpm_build_root. I'd have to install the venv into the ACTUAL location on the build server it was going to be when we deploy (install rpm). This is not ideal for a lot of reasons you may be able to guess (collisions with other applications, other stuff running on build servers etc).

I don't want to run setup.py on my production box and download packages at install time. I want to make sure all is well, have everything downloaded and packaged before deploy happens.

The closest thing I have found is dh-virtualenv from this so question. This looks promising and from what I can tell installs directly into the final directory (not a temp build). It cleans up after itself but still seems bad practice. Is there a better way? Am I missing something? Seems I'm stuck doing it the spotify way.

Upvotes: 4

Views: 4980

Answers (1)

Chris Maes
Chris Maes

Reputation: 37712

I don't know what is the best way, but your idea is very close to what we do at our company:

  • create a relative virtualenv (that can be relocated and is still functional). I have rendered the script public in this gist
  • then package the whole virtualenv, changing also the shebang of scripts such that they are invoked with the correct virtualenv.

We would have something like this in our spec files:

%build
create-relative-virtualenv -p . -v 3.6 -r Pipfile

%install
# Create directories
install -d -m 0755 "${RPM_BUILD_ROOT}/usr/lib/application"

cp -r virtualenv ${RPM_BUILD_ROOT}/usr/lib/application/
cp script.py ${RPM_BUILD_ROOT}/usr/lib/application/
# set the correct shebang for script
sed -i "1s@.*@#\!/usr/lib/application/virtualenv/bin/python3@" ${RPM_BUILD_ROOT}/usr/lib/application/script.py
ln -s /usr/lib/application/script.py ${RPM_BUILD_ROOT}/usr/bin/script

%check
./virtualenv/bin/python3 -m coverage run --branch tests/run_tests.py

%files
/usr/lib/application
/usr/bin/script

Nice detail here is that the tests run with the same virtualenv that is packaged.

Upvotes: 3

Related Questions