CODEWITHSUNDEEP
python
Srini
Srini

Reputation: 1639

Including a python library with my script

My system administrator will not allow global installation of python packages. I'm writing a script that people will invoke to perform certain actions for them. The script I'm writing needs certain libraries like sqlalchemy and coloredlogs. I am however allowed to install python libs any local folder. i.e not site-packages.

How would I go about installing the libs in the same folder as the script so that the script has access to them?

My folder hierarchy is like so

script_to_invoke.py
scriptpack/
    bin/
        coloredlogs
        coloredlogs.egg
        ...
    utils/
        util1.py
        util2.py

(all the folders indicated have an __init__.py)

What I've tried so far:

within script_to_invoke.py I use

from scriptpack.utils invoke util1 # no problem here
from scriptpack.bin import coloredlogs # fails to find the import

I've looked at some other SO answers abut I'm not sure how to correlate them with my problem.

Upvotes: 1

Views: 3480

Answers (3)

ShadowRanger
ShadowRanger

Reputation: 155363

If you might want to try packaging up everything as a zipapp. Doing so makes a single zip file that acts as a Python script, but can contain a whole multitude of embedded packages. The steps to make it are:

  1. Make a folder with the name of your program (testapp in my example)
  2. Name your main script __main__.py and put it in that folder
  3. Using pip, install the required packages to the folder with --target=/path/to/testapp
  4. Run python3 -mzipapp testapp -p='/usr/bin/env python3' (providing the shebang line is optional; without it, users will need to run the package with python3 testapp.pyz, while with the shebag, they can just do ./testapp.pyz)

That creates a zip file with all your requirements embedded in it alongside your script, that doesn't even need to be unpacked to run (Python knows how to run zip apps natively). As a trivial example:

$ mkdir testapp
$ echo -e '#!/usr/bin/python3\nimport sqlalchemy\nprint(sqlalchemy)' > __main__.py
$ pip3 install --target=./testapp sqlalchemy
$ python3 -mzipapp testapp -p='/usr/bin/env python3'
$ ./testapp.pyz
<module 'sqlalchemy' from './testapp.pyz/sqlalchemy/__init__.py'>

showing how the simple main was able to access sqlalchemy from within the same zipapp. It's also smaller (thanks to the zipping) that distributing the uncompressed modules:

$ du -s -h testapp*
13M     testapp
8.1M    testapp.pyz

Upvotes: 1

Srini
Srini

Reputation: 1639

I figured it out!

Python had to be directed to find the .egg files This can be done by either

  1. Editing the PYTHONPATH var BEFORE the interpreter is started (or)
  2. Appending the full path to the eggs to the sys path

Code Below:

import sys

for entry in [<list of full path to egg files in bin dir>]:
    sys.path.append(str(entry))

# Proceed with local imports

Upvotes: 2

puzzlepalace
puzzlepalace

Reputation: 660

You can install these packages in a non-global location (generally in ~/.local/lib/python<x.y>) using the --user flag, e.g.:

pip install --user sqlalchemy coloredlogs

That way you don't have to worry about changing how imports work, and you're still compliant with your sysadmins policies.

Upvotes: 0

Related Questions