Adding packages to Python "embedded" installation for Windows

Question

From the documentation:

The embedded distribution is a ZIP file containing a minimal Python environment.

Sounds great! The 64-bit Windows embedded v3.6.5 of Python is only 13MB. As an alternative to compiling, I would like to zip some python scripts together with the minimum needed to run them on a Win10 machine that doesn't have Python installed.

Now, I almost always need to import additional packages to provide functionality. But I can't determine how I should do this if I want to send out a python script together with this embedded version of Python. For example, if my script uses numpy, how can I include that library in this "embed?" I.e., so that on any Win10 machine I can unzip the single deployment file and immediately execute my scripts?

(A regular pip install numpy appears to create a Lib subdirectory that's over 50MB! But for an "embedded" deployment I don't need any support for debugging or whatever else is encompassed in that mass of files.)

Answer 1

At least with the latest Python versions (tested on 3.8 and 3.11) this appears to work fine:

• Download the Windows embeddable package you need from the official site, and extract it.

  PS> Invoke-WebRequest -Uri https://www.python.org/ftp/python/3.11.1/python-3.11.1-embed-amd64.zip -OutFile python-3.11.1-embed-amd64.zip
  PS> Expand-Archive .\python-3.11.1-embed-amd64.zip
  PS> cd .\python-3.11.1-embed-amd64
  

• Open the python3xx._pth file corresponding to your version (e.g. python311._pth for Python 3.11), and make sure the following import line is uncommented. This will automatically add site directories used by pip to the Python path:

  # Uncomment to run site.main() automatically
  import site
  

  In Powershell this can be automated by running:

  PS> Add-Content -Path .\python311._pth -Value 'import site'
  

• Download the official pip bootstrap script, e.g. with Powershell:

  PS> Invoke-WebRequest -Uri https://bootstrap.pypa.io/get-pip.py -OutFile get-pip.py
  

• Run the downloaded script (make sure you're using the correct Python executable in the current directory .\):

  PS> .\python.exe get-pip.py
  Collecting pip
    Using cached pip-22.3.1-py3-none-any.whl (2.1 MB)
  Collecting setuptools
    Downloading setuptools-66.0.0-py3-none-any.whl (1.3 MB)
       ---------------------------------------- 1.3/1.3 MB 16.0 MB/s eta 0:00:00
  Collecting wheel
    Using cached wheel-0.38.4-py3-none-any.whl (36 kB)
  Installing collected packages: wheel, setuptools, pip
    WARNING: The script wheel.exe is installed in 'C:\Users\xxxx\Downloads\python-3.11.1-embed-amd64\Scripts' which is not on PATH.
    Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
    WARNING: The scripts pip.exe, pip3.11.exe and pip3.exe are installed in 'C:\Users\xxxx\Downloads\python-3.11.1-embed-amd64\Scripts' which is not on PATH.
    Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  Successfully installed pip-22.3.1 setuptools-66.0.0 wheel-0.38.4
  

• Run pip from the .\Scripts directory:

  PS> .\Scripts\pip.exe list
  Package    Version
  ---------- -------
  pip        22.3.1
  setuptools 66.0.0
  wheel      0.38.4
  
  PS> .\Scripts\pip.exe install numpy
  Collecting numpy
    Downloading numpy-1.24.1-cp311-cp311-win_amd64.whl (14.8 MB)
       ---------------------------------------- 14.8/14.8 MB 11.9 MB/s eta 0:00:00
  Installing collected packages: numpy
    WARNING: The script f2py.exe is installed in 'C:\Users\xxxx\Downloads\python-3.11.1-embed-amd64\Scripts' which is not on PATH.
    Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  Successfully installed numpy-1.24.1
  
  PS> .\python.exe -c 'import numpy; print(numpy.__version__)'
  1.24.1
  

Answer 2

There is a way to extend Python embedded installation. I managed to create Flask-ready package, that I can just unzip on target machine and run code. The trick is to install EXACT same python version (normal full blown python) as your target embedded small python. Not only version but x86, x64 has to match as well.

Then install modules from pip on normal python, go to NormalPython\Lib\site-packages and copy all new files that appear after installing the package to EmbeddedPython\Lib finally add Lib and Lib\site-packages to pythonXX._pth inside Embedded python folder.

It's extremely important to fully test your application in case you miss some package. Also this would not work for packages that also add .exe to Scripts folder. You could still probably copy the exe's to Script folder and add it to path which could do the trick.

Answer 3

There is a similar trick.

1. Install pip in the embedded version:

   curl -sSL https://bootstrap.pypa.io/get-pip.py -o get-pip.py

   python get-pip.py

2. Edit pythonXX._pth by adding

   Lib

   Lib\site-packages

3. Use pip to install other packages

Answer 4

There is a way to extend Python embedded installation. The trick is to install the same python version( I will call it NormalPython) as your target embedded python(I will call it EmbeddedPython). The version as well as architecture has to match exactly.

You then install modules from pip on NormalPython. You can find pip on NormalPython\Scripts.

Go to NormalPython\Lib\site-packages and copy all files that appear after installing whatever you want to install through pip to EmbeddedPython\Lib\

Then add Lib to pythonXX._pth inside Embedded python folder.

This worked for me on windows 10 after downloading the latest python embed and python install through https://www.python.org/downloads/windows/ . I used Michal Rawluk's answer to do it, but it was somewhat hard to understand so I am trying to explain it a bit differently.