CODEWITHSUNDEEP
pythonmacos
OregEmber
OregEmber

Reputation: 45

Organizing downloaded python libraries

I've noticed that my python libraries and other dependencies are stored at various places around my directory hierarchy. For example,

A recent failure to upgrade one of (and that my $PATH var now contains 7 directories) makes me believe that I need to somehow reorganize these files.

1) When 'pip'ing or 'brew'ing a new library, say, is there any particular directory the downloaded files should go to? (e.g. /Library/python/2.7/site-packages )

2) Are there any useful sources of how to organize one's own personal file system? (Specifically with downloaded python libraries, or more generally)

Upvotes: 2

Views: 186

Answers (1)

abarnert
abarnert

Reputation: 365617

Let's take these out of order:

1) When 'pip'ing or 'brew'ing a new library, say, is there any particular directory the downloaded files should go to?

First, the downloaded files go into a temporary directory and get deleted after installation. That's automatically taken care of by both pip and brew.

For Python packages, whether installed by pip or not, you do have a choice between system-wide and user-specific installs (see User Installs), but otherwise, they just go into your Python site-packages. That location can be configured, but there's no good reason to. For Apple's pre-installed Python, it's /Library/python/2.7/site-packages. If you want more control, you shouldn't try to keep custom site-packages directories around (or multiple Python installations); use virtualenv to built separate virtual environments, and then pip will install into whichever environment is currently active, instead of installing into your normal site-packages.

For programs and libraries that you install with brew, they get built into "kegs", isolated directories under /usr/local/Cellar, and then anything useful gets linked from there into /usr/local/bin, /usr/local/lib, etc. If you want to know more, the Homebrew docs explain it all in detail. This of course includes libraries that you install as dependencies for Python packages. For example, if you brew install libxslt so you can pip install lxml, brew has no idea why you're installing libxslt, it just builds it in a keg and links it into /usr/local/lib like anything else.

Either way, you should not try to organize things. Both pip and brew know where they install things, and if you screw with them after the fact, uninstall, update, etc. will probably break.

and a recent failure to upgrade one of (and that my $PATH var now contains 7 directories) makes me believe that I need to somehow reorganize these files.

Your $PATH has nothing to do with where Python looks for libraries. That's where your shell looks for executable programs. What Python looks in is sys.path, which is a completely different thing.

I've noticed that my python libraries and other dependencies are stored at various places around my directory hierarchy

No, they're really not. Every Python library you install goes into the site-packages directory.

So, what about all that other stuff on sys.path? Well, there's the Python standard library (/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7 and/or /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7.zip), and extra modules that Apple pre-installs (/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python), and various subdirectories (and zipfiles and eggfiles) within those two locations, and of course the current working directory ('.' or ''), and that should be it.

(i.e. /usr/local/python, /Library/python/2.7/site-packages, or /usr/bin/python..)

/usr/bin/python is not a place where libraries are stored. It's not a directory at all; it's an executable. It's the program that gets run when you type python at the terminal, or run a script that starts with #!/usr/bin/python or #!/usr/bin/env python.

/usr/local/python shouldn't exist at all. It's not part of the standard path for Apple's pre-installed Python, the official Python.org installation, or a Homebrew installation. If you've created this and added it to your sys.path… well, don't do that if you don't want to do that.

If you meant /usr/local/bin/python, then (a) that's an executable, not a place for libraries, and (b) it means you've installed a second Python alongside the Apple pre-installed one, and if you can't keep them straight, you should probably get rid of it.

Upvotes: 2

Related Questions