CODEWITHSUNDEEP
pythonvirtualenv
Henry Henrinson
Henry Henrinson

Reputation: 5412

Make virtualenv share already existing site-packages?

My layout is as follows:

I have various different python projects under ~/projects, each with the following structure:

~/projects/$project_name/env                      #This is the virtualenv
~/projects/$project_name/scripts                  #This is where the code actually lives
~/projects/$project_name/scripts/requirements.txt #This helps keep track of this project's dependencies

Now, this setup works great as it does the following:

  1. Each project has it's own dependencies in its corresponding env
  2. I can easily redeploy this project somewhere else by cloning the scripts file, creating a new virtualenv and doing pip install -r requirements.txt

The main downside of this setup is that I have multiple copies of the same packages in multiple virtual environments. I regularly end up with a couple of hundred megs for each virtual environment.

My question is:

Is there a way to share packages between multiple virtualenvs?

Things I've tried and do not work:

virtualenv --system-site-packages. This makes the system-wise packages available in the virtualenv but:

  1. it makes it impossible to get a list of specific dependencies
  2. I can't have multiple versions of the same dependency installed (e.g. pandas 0.16 vs pandas 0.15) which I need, as different projects have different needs.

virtualenv --extra-search-dir=/path/to/dist only works for pip, AFAICT, so not good for me.

Upvotes: 2

Views: 1434

Answers (1)

Neil
Neil

Reputation: 3311

Scrap the comment, maybe I do know an answer. It appears as though Anaconda's package management system does use symlinks. So that would basically be a virtualenv but with the feature you want. See here: How to free disk space taken up by (ana)conda?

That said, there's a large initial harddisk cost to using Conda, so investigate a bit more and decide if it will work for you.

Upvotes: 1

Related Questions