CODEWITHSUNDEEP
pythonsetuptoolseasy-install
jterrace
jterrace

Reputation: 67073

Eggs in path before PYTHONPATH environment variable

If I have packages installed from easy_install, the eggs are prepended to sys.path before the items in the PYTHONPATH variable.

For example, if I have an egg package called foo installed as well as a package called foo in the current directory, and then do this:

PYTHONPATH="." python
>>> import foo

This will use the egg version of foo instead of the local directory. Inspecting sys.path shows that eggs are placed before items from PYTHONPATH. This seems broken. Is there any way to override this behavior?

Upvotes: 16

Views: 12510

Answers (3)

bsb
bsb

Reputation: 1937

Consider using the -S command-line option to suppress *.pth processing:

python -c 'import sys; print("\n".join(sys.path))'
python -S -c 'import sys; print("\n".join(sys.path))'

https://docs.python.org/3/library/site.html#site.main

You can also use -S with site.main() to delay *.pth processing until runtime, say to capture the original sys.path for appending:

export PYTHONPATH=$(
  PYTHONPATH='' \
  python -c 'import sys; \
    sys.path.extend(sys.argv[1:]); old=list(sys.path); \
    import site; site.main(); \
    [ old.append(p) for p in sys.path if p not in old ]; \
    sys.path=old; \
    print ":".join(sys.path)' \
  $EXTRA_PATH $ANOTHER_PATH)

python -S ... # using explicit PYTHONPATH
  • Start from explicit empty PYTHONPATH
  • Append to sys.path explicitly with extend
  • Import site and call site.main()
  • Append new paths to old path and then install it in sys.path
  • Print with ":" for PYTHONPATH
  • python -S is desirable for later runs only using $PYTHONPATH
  • python -S may or may not be desirable while setting PYTHONPATH (depending on if you need sys.path expanded before extending)

Upvotes: 2

nak
nak

Reputation: 107

I have done something like the following to prepend to the system path when running a top-level python executable file:

import sys
sys.path = ["<your python path>"] + sys.path

Often, the "<your python path>" for me involves use of the __file__ attribute to do relative look up for a path that includes the top-level module for my project. This is not recommended for use in producing, eggs, though I don't seem to mind the consequences. There may be another alternative to __file__.

Upvotes: 1

samplebias
samplebias

Reputation: 37919

Unfortunately this is done with a hard-coded template deep inside setuptools/command/easy_install.py. You could create a patched setuptools with an edited template, but I've found no clean way to extend easy_install from the outside.

Each time easy_install runs it will regenerate the file easy_install.pth. Here is a quick script which you can run after easy_install, to remove the header and footer from easy_install.pth. You could create a wrapper shell script to run this immediately after easy_install:

#!/usr/bin/env python
import sys
path = sys.argv[1]
lines = open(path, 'rb').readlines()
if lines and 'import sys' in lines[0]:
    open(path, 'wb').write(''.join(lines[1:-1]) + '\n')

Example:

% easy_install gdata
% PYTHONPATH=xyz python -c 'import sys; print sys.path[:2]'
['', '/Users/pat/virt/lib/python2.6/site-packages/gdata-2.0.14-py2.6.egg']

% ./fix_path ~/virt/lib/python2.6/site-packages/easy_install.pth
% PYTHONPATH=xyz python -c 'import sys; print sys.path[:2]'
['', '/Users/pat/xyz']

For more clarification, here is the format of easy-install.pth:

import sys; sys.__plen = len(sys.path)
./gdata-2.0.14-py2.6.egg
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

The two import sys lines are the culprit causing the eggs to appear at the start of the path. My script just removes those sys.path-munging lines.

Upvotes: 13

Related Questions