CODEWITHSUNDEEP
python-3.xdockerdocker-composegitlab
r.chub
r.chub

Reputation: 11

How to install custom modules from another repo using docker

I've got some problem with installing custom modules from another repo to this project_name. It's all good when I use PyCharm. So, how can I install custom modules from another repo using ssh deploy keys in docker?

Structure of project:

project_name
|-core
|  |-models
|  |  |- __init__.py
|  |  |- ...py
|  |-start.py
|  |-Dockerfiles
|  |  |- Dockerfile
|  |  |- id_rsa
|  |  |- id_rsa.pub
|  |  |- ssh.config
|  |-start.py
|- config.py 
|- setup.py

Setup.py

from setuptools import setup, find_packages
from os.path import join, dirname

setup(
   name='core',
   version='0.1',
   url='https://gitlab.ru/username/repo_name.git',
   install_requires=['custom_module>=0.1',
                     'sqlalchemy>=1.2.2',
                     'redis>=2.10.0',
                     'hiredis>=0.2',
                     'python-socketio',
                     'aioredis',
                     'psycopg2',
                     'alembic',
                     'flask',
                     'flask-admin',
                     'flask_basicauth'],
  dependency_links=['git+ssh://[email protected]/username/custom_module.git'],
  include_package_data=True,
  packages=find_packages(),
  entry_points={
     'console_scripts':
         ['core = core.start']
      }
)

Dockerfile:

FROM python:3.6
RUN mkdir -p /var/project_name
RUN mkdir -p ~/.ssh

RUN apt install -y  openssh-client git

COPY . /var/project_name/
COPY Dockerfiles/id_rsa /var/HEAVEN-CORE/id_rsa
COPY Dockerfiles/id_rsa.pub /var/HEAVEN-CORE/id_rsa.pub
COPY Dockerfiles/ssh.config /var/HEAVEN-CORE/ssh.config

RUN cat /var/project_name/id_rsa > ~/.ssh/id_rsa
RUN cat /var/project_name/id_rsa.pub > ~/.ssh/id_rsa.pub
RUN cat /var/project_name/ssh.config > ~/.ssh/config

RUN eval `ssh-agent -s` && chmod 0600 ~/.ssh/id_rsa  && ssh-add 
~/.ssh/id_rsa && cd /var/project_name  && pip3 install 
git+ssh://[email protected]/username/repo_name.git 

ENV PYTHONPATH $PYTHONPATH:/var/project_name

WORKDIR /var/project_name

I has exception in terminal:

Collecting git+ssh://[email protected]/username/repo_name.git
Cloning ssh://[email protected]/username/repo_name.git to 
/tmp/pip-req-build-0m86uyex

Warning: Permanently added 'gitlab.ru,62.76.114.78' 
(ECDSA) to the list of known hosts.

Collecting git+ssh://[email protected]/username/custom_module.git
Cloning ssh://[email protected]/username/custom_module.git to 
/tmp/pip-req-build-zu7jupsg

Collecting custom_module>=0.1 (from core==0.1)
Could not find a version that satisfies the requirement 
custom_module>=0.1 (from core==0.1) (from versions: )

No matching distribution found for custom_module>=0.1 (from core==0.1) 
ERROR: Service 'core' failed to build: The command '/bin/sh -c eval 
`ssh-agent -s` && chmod 0600 ~/.ssh/id_rsa  && ssh-add ~/.ssh/id_rsa 
&& cd /var/project_name  && pip3 install 
git+ssh://[email protected]/username/repo_name.git' returned a 
non-zero code: 1

It's wrong when I want to use pip install git+ssh://[email protected]:username/repo_name.git like in Gitlab SSH URL. Error: Could not read from remote repository.

Upvotes: 0

Views: 773

Answers (3)

Victor Zuanazzi
Victor Zuanazzi

Reputation: 1964

I had a similar issue, but with a different project structure. I am adding my solution here for the sake of completeness.

If you have the following file structure:

project_name
|-core
|  |-models
|  |  |- __init__.py
|  |  |- ...py
|  |-start.py
|-requierements
|  |- prod.txt
|  |- dev.txt
|- Dockerfile
|- Makefile
|- ...

Where the dependencies are stored on the files prod.txt and dev.txt. You can also create an dependency to a branch with the following statement inside either txt file:

custom_module @ git+ssh://gitlab.skytracking.ru/username/custom_module.git@branch#egg=custom_module-0

Upvotes: 0

r.chub
r.chub

Reputation: 11

It might be help someone else. I solved this issue

install_requires=['custom_module',
                  ...
                  ],
dependency_links=['git+ssh://gitlab.skytracking.ru/username/custom_module.git@branch#egg=custom_module-0']

@branch = it's your branch (master or smth else). At the end add -0, not a version.

Add this you can install custom modules from private repo

Upvotes: 1

anamar
anamar

Reputation: 380

Try echoing gitlab host key into known hosts: 

RUN ssh-keyscan gitlab.ru >> ~/.ssh/known_hosts

More info on https://docs.gitlab.com/ee/ci/ssh_keys/.

If you're installing a module from a public repository, you could try over https instead of ssh: 

pip3 install git+https://gitlab.ru/username/repo_name.git

Upvotes: 0

Related Questions