CODEWITHSUNDEEP
pythonvirtualenv
hazrmard
hazrmard

Reputation: 3661

How to share packages between virtual environments using conda or virtualenv?

I have multiple python projects, each with its own virtual environment. Additionally I have a global python environment. I want to share the same package files between virtual/global environments without copying them multiple times (where possible).


Environment        Packages
----------------------------------------
GLOBAL:      A    B    C    D    
VENV_1:      A    B    C         E
VENV_2:                C    D         F

So, in this example:

I tried using virtualenv with the --system-site-packages flag, but that just makes all global packages accessible in the virtual environment. So when I export my environment using pip freeze it will contain unnecessary packages.

How can I create a virtual environment with select packages, such that if a package is globally installed, a new copy of files for that package won't be made locally? Is this possible using conda or virtualenv or any other tool?

I am using Windows 10 with python 3.6.

Upvotes: 5

Views: 3506

Answers (2)

Mostafa Wael
Mostafa Wael

Reputation: 3846

  • In pip you can cache downloads in ~/.pip/cache so it won't need to download them again next time by adding the following to $HOME/.pip/pip.conf:
[global]
download_cache = ~/.pip/cache
  • In conda, you can use a shared package cache by creating a directory on your system where the shared users have read and write access.

Then, for each user who will have access, edit the .condarc file found in their home directory with the following entry, specifying the full path to that shared directory:

pkgs_dirs:
    - /path/to/shared_directory

Windows - C:\Users\username.condarc

macOS and Linux - /home/username/.condarc

Verify the package cache by running conda info.

Upvotes: 0

SteveJ
SteveJ

Reputation: 3323

With PyCharm you can associate projects, and you could probably manipulate the PYTHONPATH variables in your venv/Source/activate.bat file but I think you are barking up the wrong tree.

I believe you would be much better off setting up a private PyPi Server (example). In so doing, you could pip-install into your new project - with the added advantage of version control and other goodness. Otherwise, you are really just undoing much of the encapsulation benefits that virtual environments give you in the first place.

Upvotes: 0

Related Questions