CODEWITHSUNDEEP
pythondockergitlabdockerfile
eddy
eddy

Reputation: 103

Installing python package from private gitlab repo in Dockerfile

I'm currently trying to install python packages from a private gitlab repo. Unfortunately, I get problems with the credentials. Is there any way to install this package without writing my credentials into the Dockerfile or adding my personal ssh key into it?

Dockerfile:

FROM python:3.9.12-buster AS production

RUN apt-get update && apt-get install -y git

COPY ./requirements.txt /app/requirements.txt

RUN pip install -r /app/requirements.txt

requirements.txt:

fastapi
uvicorn
cycler~=0.10.0
networkx
python-multipart
git+https://gitlab.private.net/group/private-repo.git@commit_hash#egg=foo

Error message:

#10 3.760   Cloning https://gitlab.private.net/group/private-repo.git (to revision commit_hash) to /tmp/pip-install-q9wtmf_q/foo_commit_hash     
#10 3.769   Running command git clone --filter=blob:none --quiet https://gitlab.private.net/group/private-repo.git /tmp/pip-install-q9wtmf_q/foo_commit_hash
#10 4.039   fatal: could not read Username for 'https://gitlab.private.net/group/private-repo.git': No such device or address
#10 4.060   error: subprocess-exited-with-error

Upvotes: 2

Views: 4095

Answers (2)

Pravesh Chapagain
Pravesh Chapagain

Reputation: 68

So I also had to install my dependencies from private package repository for my python project. This was the Dockerfile I used for building my project.

ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
RUN apt-get update &&\
    apt-get install -y binutils libproj-dev gettext gcc libpq-dev python3-dev build-essential python3-pip python3-setuptools python3-wheel python3-cffi libcairo2 libpango-1.0-0 libpangocairo-1.0-0 libgdk-pixbuf2.0-0 libffi-dev shared-mime-info

RUN pip config set global.extra-index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.com/simple/  
# you need to configure pip to pull packages from remote private repository.
# for gitlab you require personal access token to access them with read permissions


COPY . /code/

RUN --mount=type=cache,target=/root/.cache pip install -r requirements.txt

RUN --mount=type=cache,target=/root/.cache pip install -r /code/webapi/requirements.txt

WORKDIR /code/webapi

ENTRYPOINT /code/webapi/entrypoint.sh

Upvotes: 0

sytech
sytech

Reputation: 40861

Generally speaking, you can use multi-stage docker builds to make sure your credentials don't stay in the image.

In your case, you might do something like this:

FROM python:3.9.12-buster as download
RUN apt-get update && apt-get install -y git
RUN pip install --upgrade pip wheel
ARG GIT_USERNAME
ARG GIT_PASSWORD

WORKDIR /build
COPY requirements.txt .
# add password to requirements file
RUN sed -i -E "s|gitlab.private.net|$GIT_USERNAME:[email protected]|" requirements.txt

# download dependencies and build wheels to /build/dist
RUN python -m pip wheel -w /build/dist -r requirements.txt

FROM python:3.9.12-buster as production
WORKDIR /app
COPY --from=download /build/dist /wheelhouse
# install dependencies from the wheels created in previous build stage
RUN pip install --no-index /wheelhouse/*.whl

COPY . .
# ... the rest of your dockerfile

In GitLab CI, you might use the build command like this:

script:
  # ...
  - docker build --build-arg GIT_USERNAME=gitlab-ci-token --build-arg GIT_PASSWORD=$CI_JOB_TOKEN -t $CI_REGISTRY_IMAGE .

Then your image will be built and the final image won't contain your credentials. It will also be smaller since you don't have to install git :)

As a side note, you can simplify this somewhat by using the GitLab PyPI package registry.

Upvotes: 1

Related Questions