Create Executable from a Python module with Pyinstaller or Cython

Question

I want to convert to an executable this python 2.7 project that has a module structure:

(.venv) ip-192-168-22-127:indictrans loretoparisi$ tree -L 1
.
├── __init__.py
├── __init__.pyc
├── __init__.spec
├── _decode
├── _utils
├── base.py
├── build
├── mappings
├── models
├── script_transliterate.py
├── tests
├── transliterator.py
└── trunk

I'm using pyinstaller for this at first stage I'm just doing:

pyinstall --onefile __init__.py

and I get a executable built:

192 INFO: PyInstaller: 3.3.1
192 INFO: Python: 2.7.10
201 INFO: Platform: Darwin-17.7.0-x86_64-i386-64bit
202 INFO: wrote /Users/loretoparisi/Documents/Projects/AI/indic-trans/indictrans/__init__.spec
208 INFO: UPX is not available.
209 INFO: Extending PYTHONPATH with paths
['/Users/loretoparisi/Documents/Projects/AI/indic-trans',
 '/Users/loretoparisi/Documents/Projects/AI/indic-trans/indictrans']
210 INFO: checking Analysis
218 INFO: checking PYZ
223 INFO: checking PKG
224 INFO: Bootloader /Users/loretoparisi/Documents/Projects/AI/indic-trans/.venv/lib/python2.7/site-packages/PyInstaller/bootloader/Darwin-64bit/run
224 INFO: checking EXE
225 INFO: Rebuilding out00-EXE.toc because __init__ missing
225 INFO: Building EXE from out00-EXE.toc
225 INFO: Appending archive to EXE /Users/loretoparisi/Documents/Projects/AI/indic-trans/indictrans/dist/__init__
230 INFO: Fixing EXE for code signing /Users/loretoparisi/Documents/Projects/AI/indic-trans/indictrans/dist/__init__
234 INFO: Building EXE from out00-EXE.toc completed successfully.

But when I run it I get an import error

Traceback (most recent call last):
  File "indictrans/__init__.py", line 9, in <module>
ValueError: Attempted relative import in non-package
[30629] Failed to execute script __init__

This library is built using Cython via cythonize setup, so another option would be to build a executable embedded module using the --embed Cython option.

My setup.py is the following:

#!/usr/bin/env python

import os

from setuptools import setup
from setuptools.extension import Extension
from Cython.Build import cythonize

import numpy


os.environ['PBR_VERSION'] = '1.2.3'
os.environ['SKIP_WRITE_GIT_CHANGELOG'] = '1'
os.environ['SKIP_GENERATE_AUTHORS'] = '1'


extensions = [
    Extension(
        "indictrans._decode.beamsearch",
        [
            "indictrans/_decode/beamsearch.pyx"
        ],
        include_dirs=[numpy.get_include()]
    ),
    Extension(
        "indictrans._decode.viterbi",
        [
            "indictrans/_decode/viterbi.pyx"
        ],
        include_dirs=[numpy.get_include()]
    ),
    Extension(
        "indictrans._utils.ctranxn",
        [
            "indictrans/_utils/ctranxn.pyx"
        ],
        include_dirs=[numpy.get_include()]
    ),
    Extension(
        "indictrans._utils.sparseadd",
        [
            "indictrans/_utils/sparseadd.pyx"
        ],
        include_dirs=[numpy.get_include()]
    )

]

setup(
    setup_requires=['pbr'],
    pbr=True,
    ext_modules=cythonize(extensions)
)

While it's easy to compile a singe python file with the --embed option, see here for more about this, I do not know how to use the --embed option in the setup.py in order to get rid of all the dependencies in the project.

Answer 1

I'm going to refer to the root package directory with the setup.py file as the package directory, which is indic-trans in your case. I'm referring to the first level source directory as the module directory, which is indic-trans/indictrans in your case.

---

If I understand your setup correctly, you have an issue because of trying to create an .exe on a script within the module directory rather than the package directory. This makes the current directory the internal module directory, and relative references will not work.

The way I resolved this in a similar situation was to create a run.py script in the main package directory (same folder as setup.py) which imports the package and then runs whatever script you want to run inside the module directory.

You didn't post the __init__.py file (by the way, it is generally frowned upon to have any real code in __init.py__...), so I'll assume for the below that you want to run the main() function in the base.py script. In that case, make run.py something like:

# This will import everything from the module __init__.py based on it's "all" setup
# You likely want to limit it to just what is needed or just the script you are targeting
# Or you can use the "import indictrans" format and let __init__.py handle it
from indictrans import * 

def main():
    indictrans.base.main()

if __name__ == '__main__':
    main()

Now you can run just run.py from the command line or use it as a debug target to run your package.

Note that with PyInstaller you can target run.py (or run.spec) and change the name to something else with the --name argument.

pyinstaller --onefile --name indictrans run.spec

which will create indictrans.exe in the dist directory.

---

Running your package as a module

Note that in the module directory, you can also create a __main__.py file that is basically as copy of run.py and does the same thing, but will be run as a module by the Python executable if it is installed locally.

i.e. python -m indictrans will run your package using __main__.py as the entrypoint.