Install pip and virtualenv, a chicken and the egg dilemma?

Question

I've already been using pip and virtualenv (and actually sometimes still prefer a well organized combination through an SVN repository, wise usage of svn:externals, and dynamic sys.path).

But this time for a new server installation I'd like to do things in the right way.

So I go to the pip installation page and it says:

The recommended way to use pip is within virtualenv, since every virtualenv has pip installed in it automatically. This does not require root access or modify your system Python installation. [...]

Then I go to the virtualenv installation page and it suggests:

You can install virtualenv with pip install virtualenv, or the latest development version with pip install virtualenv==dev. You can also use easy_install [...]

And pip is supposed to replace easy_install, of course :)

Granted, they both explain all alternative ways to install.

But... which one should go first? And should I favor systemwide pip or not?

I see a main reason to ponder over, but there might be others

• do I want to facilitate life for all users of the box, or is this a server targeted to one single user running some services?

If I want everybody to have a virtual env available I might just install a system wide pip (eg. with ubuntu do sudo aptitude install python-pip then use it to install virtualenv sudo pip install virtualenv).

edit another reason to ponder over: virtualenvwrapper install instructions (but not the docs) say:

Note In order to use virtualenvwrapper you must install virtualenv separately.

not exactly sure what "separately" mean there (i never noticed).

Otherwise, which one should go first, and does it really make a difference or not?

Related:

The closest question (and answers) is the first of the following (in particular see @elarson answer), the second looks overly complicated:

• What is the official "preferred" way to install pip and virtualenv systemwide?
• What's the proper way to install pip, virtualenv, and distribute for Python?
• Step by step setting up python with pip and virtualenv?
• System PIP instead of virtualenv PIP by default?

but I feel it all fail at answering my question in full: systemwide vs. local, but also should pip or virtualenv go first (and why do they send each one to the other to start with!!!)

Answer 1

Not decided yet on the definitive solution, but that's what's going on right now partly based on @nutjob comments (no, I haven't switched to buildout for the time being but I will take some time for it later on!)

I have a big, powerful server with quite a lot of django applications. I mostly use gunicorn + supervisord.

These requirements dictate the following, but slightly different settings will probably make it different.

1. I have logically organized these applications in clusters, based on code/functions similarities; each cluster will likely have the same version of common python libs (pylibmc, pillow, etc.)
2. I create a few users, one for each cluster
3. For each user I install a virtualenv (with the same name as the user, for my own choice) by downloading virtualenv.py and running python virtualenv.py VIRTUALENVNAME; I don't install virtualenv wrapper
4. I edit that user .bashrc so that the virtualenv is immediately loaded
5. I install the common packages with pip, but more specific packages are added directly through sys.path This allows me an even faster deployment, because of the folder/versioning project structure that carries all most of the required libraries there. I find this more clear, faster to deploy, and easier to edit. Mileage for your individual cases might vary.

This means each user has is own, single virtualenv.

Should I start all over I would probably move to a buildout or similar solution (I'm not a big fan of what happens when you mix supervisor and virtualenv), but I'm right now pretty happy with this one...

Answer 2

tl;dr Answer would be VirtualEnv first. You can have two of them each for Python version 2.x and 3.x

[edit]

I am really doubtful if installing (there is no install, you merely download and execute a script) VirtualEnv system wide/per-user even matters. The whole point of Using VirtualEnv is to create isolated development sandboxes so that the libraries from one project doesn't conflict with each other. For example you can a Python 2.x project using Beautiful-soup Version < 4.x and A Python 3.x project Using Beautiful-soup Version 4.0 in two different Virtual Environments.

How you get VirtualEnv script on your system doesn't really matter, and since once you have it and pip is self contained within VirtualEnv, it just makes sense to get VirtualEnv first. Also once you are in with python, you would have many projects, and for each, the recommended way would be to have a Virtual Environment, and then install dependencies via pip. You can later do "pip freeze > requirements.txt" and then a "pip install requirements.txt" to simply replicate your exact libraries across two systems [say dev and production] and so on...

Answer 3

Half of one, six dozen of another. (Let that sink in. Ha ha.)

But more seriously, do you honestly have multiple users on your system? These days, even Linux hosts tend to be for a single user, and where there are multiple user IDs they tend to be servers that run multiple processes under various quarantined user IDs. Given that, making life easier for all users isn't quite so relevant.

On the other hand, multiple services each using Python may have conflicting requirements, rare as it may be that it boils down to even a required version of pip. Given that, I'd tend to prefer a global installation of virtualenv in order to make pristine quasi-installations of Python.

Yet I'd like to point out one other idea: Buildout, http://www.buildout.org/

Buildout does the same thing as virtualenv but by taking a remarkably different approach. You write a buildout configuration file (buidout.cfg) that lists what your various eggs are and how they'll be connected, and specify settings of "buildout recipes" that set up specialized situations (like a Django deployment, a Buildbot server, a Plone website, a Google app engine app, etc.).

You then use your system Python to bootstrap the buildout, run it, and it generates an isolated setup—like a virtualenv.

But the best part: it's repeatable. You can take the same buildout.cfg to another host and get the same setup. That's much harder to do with a virtualenv!