CODEWITHSUNDEEP
pythonpython-3.xvirtualenvpython-2.xcedar
patrick7
patrick7

Reputation: 395

Problems using Pip Install on Cedar inside a virtual environment

I having problems trying to install some packages using virtualenv on Cedar. I am following the directions found on https://docs.computecanada.ca/wiki/Python but however I keep running into one of two problems: unable to load python 3.7, or able to load python 3.7 but unable to use pip install. I am able to create the virtual environment successfully, but the problems come when I’m installing modules. I’ve tried the commands in two different orders which produced two undesirable results:

$ module load python/3.7.0
$ source ~/ENV/bin/activate
$ pip install numpy --no-index

This allows me to install numpy (or any other package), but when I try to open python using $python, I go into python 2.7.14 not 3.7.0. (Is there a command that allows me to open python 3.7.0 specifically?)

However when I tried it another way:

$ source ~/ENV/bin/activate 
$ module load python/3.7.0
$ pip install numpy --no-index

This allows me to open python 3.7.0 when I type $python but however does not allows me to use pip commands.

I also tried using pip first before using module load python/3.7.0 This would allows me to use pip install but however upon opening python, the package I have installed previously can not be found. I am aware using the second method I can also use module load SciPy-stack which contains numpy and is able to be imported into Python 3.7.0, but I need the joblib package which is not included.

I have attached 2 simple images of both results in case that helps more. I really appreciate your help. Thanks!

There I show the 2 cases. Case 2 is the picture above and case 1 is the picture below.

Upvotes: 0

Views: 769

Answers (1)

o_o
o_o

Reputation: 1

mine works in this order 1 "module load" 2. create and activate virtual environment 3. pip insall here is a sample of my job file

#!/bin/bash
#SBATCH --time=1-00:00
#SBATCH --account=def-someone
#SBATCH --job-name=jobname
#SBATCH --gres=gpu:v100l:2
#SBATCH --cpus-per-task=3
#SBATCH --mem=125G
#SBATCH --output=%x-%j.out
module load arch/avx512 StdEnv/2018.3
nvidia-smi
module load python/3.7.4
module load scipy-stack
module load nixpkgs/16.09  
module load gcc/7.3.0

module spider cuda/10.1
module spider cudnn/7.6.5
virtualenv --no-download $SLURM_TMPDIR/myenv
source $SLURM_TMPDIR/myenv/bin/activate

pip install --no-index --upgrade pip
pip install --no-index torch torchvision
pip install --no-index h5py

python main.py

Upvotes: 0

Related Questions