Python Windows embeddable package fails to run "No module named pip" "The system cannot find the path specified: 'C:\\python-3.9.6-embed-amd64\\DLLs'"

Question

Python from python-3.9.6-embed-amd64.zip fails to run, various errors

There is a series of errors that occur when trying to run Python from the Windows "embeddable" package.

---

1. Downloaded the zip file
2. unzip the zip file
3. open powershell, cd to the unzipped file path (C:\python-3.9.6-embed-amd64\)
4. update pip (fails with No module named pip)

PS> .\python.exe -m pip install --update pip
C:\python-3.9.6-embed-amd64\python.exe: No module named pip

If I manually install pip using get-pip.py, I notice strange warning

The scripts pip.exe, pip3.9.exe and pip3.exe are installed in 'C:\python-3.9.6-embed-amd64\Scripts' which is not on PATH.`

from command

PS> .\python.exe get-pip.py
Collecting pip
  Downloading pip-21.2.4-py3-none-any.whl (1.6 MB)
     |████████████████████████████████| 1.6 MB 2.2 MB/s
Collecting setuptools
  Downloading setuptools-57.4.0-py3-none-any.whl (819 kB)
     |████████████████████████████████| 819 kB 2.2 MB/s
Collecting wheel
  Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, setuptools, pip
  WARNING: The script wheel.exe is installed in 'C:\python-3.9.6-embed-amd64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts pip.exe, pip3.9.exe and pip3.exe are installed in 'C:\python-3.9.6-embed-amd64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-21.2.4 setuptools-57.4.0 wheel-0.37.0

Try to install something using pip, I run into error No module named pip

PS> .\python.exe -m pip install nuxeo
C:\python-3.9.6-embed-amd64\python.exe: No module named pip

If I get past that error, I might run into this error regarding a pickle file Error: 'lib2to3\\Grammar3.6.5.final.0.pickle'

PS> .\python.exe -m pip install nuxeo
...
error: [Errno 0] Error: 'lib2to3\\Grammar3.6.5.final.0.pickle'

If I get past that error, I might run into this error The system cannot find the path specified ...\\DLLs

PS> .\python.exe ..\script.py
...
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\python-3.9.6-embed-amd64\\DLLs'

How do I get the portable "embed" Windows Python package to run?

Answer 1

A PowerShell script that implements this answer (installs the embed.zip and adjust files) is at PythonEmbed4Win.ps1 which includes improvements not in this Answer.

---

About

Embedded Python (Windows embeddable package zip file) will not find installed packages and modules . It requires a few extra steps to fully configure.

These instructions will install, configure, and then create a new virtual python environment based on the embedded distribution.
This Python distribution and any derived virtual environments will run in a portable manner (no need for environment variable hacks, no special registry settings, etc.).

Tested with python-3.9.6-embed-amd64.zip on Windows 10 using powershell.

---

1. Download and unzip
2. adjust paths
3. Install pip using get-pip.py
4. Install virtualenv package
5. Create a new virtual environment based on the embedded distribution

---

Download and unzip

1. Download the .zip file
2. Unzip to some path (this will use example path C:\python-3.9.6-embed-amd64\). Unzipped contents will look similar to

PS > dir

    Directory: C:\python-3.9.6-embed-amd64

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         8/27/2021  12:12 PM        3406016 libcrypto-1_1.dll
-a----         8/27/2021  12:12 PM          32792 libffi-7.dll
-a----         8/27/2021  12:12 PM         690368 libssl-1_1.dll
-a----         8/27/2021  12:12 PM          32628 LICENSE.txt
-a----         8/27/2021  12:12 PM         192744 pyexpat.pyd
-a----         8/27/2021  12:12 PM         561429 python.cat
-a----         8/27/2021  12:12 PM         101608 python.exe
-a----         8/27/2021  12:12 PM          59624 python3.dll
-a----         8/27/2021  12:12 PM        4485864 python39.dll
-a----         8/27/2021  12:12 PM        2502717 python39.zip
-a----         8/27/2021  12:12 PM             79 python39._pth
-a----         8/27/2021  12:12 PM         100072 pythonw.exe
-a----         8/27/2021  12:12 PM          28904 select.pyd
-a----         8/27/2021  12:12 PM        1538792 sqlite3.dll
-a----         8/27/2021  12:12 PM        1121512 unicodedata.pyd
-a----         8/27/2021  12:12 PM          97160 vcruntime140.dll
-a----         8/27/2021  12:12 PM          37256 vcruntime140_1.dll
-a----         8/27/2021  12:12 PM          29928 winsound.pyd
-a----         8/27/2021  12:12 PM          65256 _asyncio.pyd
-a----         8/27/2021  12:12 PM          86760 _bz2.pyd
-a----         8/27/2021  12:12 PM         127208 _ctypes.pyd
-a----         8/27/2021  12:12 PM         270568 _decimal.pyd
-a----         8/27/2021  12:12 PM         179944 _elementtree.pyd
-a----         8/27/2021  12:12 PM          66280 _hashlib.pyd
-a----         8/27/2021  12:12 PM         163048 _lzma.pyd
-a----         8/27/2021  12:12 PM          40168 _msi.pyd
-a----         8/27/2021  12:12 PM          30440 _multiprocessing.pyd
-a----         8/27/2021  12:12 PM          46312 _overlapped.pyd
-a----         8/27/2021  12:12 PM          29416 _queue.pyd
-a----         8/27/2021  12:12 PM          80616 _socket.pyd
-a----         8/27/2021  12:12 PM          89832 _sqlite3.pyd
-a----         8/27/2021  12:12 PM         155368 _ssl.pyd
-a----         8/27/2021  12:12 PM          23784 _uuid.pyd
-a----         8/27/2021  12:12 PM          47336 _zoneinfo.pyd

3. Unzip C:\python-3.9.6-embed-amd64\python39.zip to C:\python-3.9.6-embed-amd64\python39

   PS> [System.IO.Compression.ZipFile]::ExtractToDirectory("C:\python-3.9.6-embed-amd64\python39.zip", "C:\python-3.9.6-embed-amd64\python39")
   

   Without these changes, you may see an error regarding pickle.

   error: [Errno 0] Error: 'lib2to3\\Grammar3.6.5.final.0.pickle'
   

   The issue is pickle cannot read files within a .zip file.

   3a. Optional: delete python39.zip

   PS> Remove-Item "C:\python-3.9.6-embed-amd64\python39.zip"
   

adjust paths

Running C:\python-3.9.6-embed-amd64\python -m pip will fail with No module named pip.

Path settings must be adjusted.

4. Change C:\python-3.9.6-embed-amd64\python39._pth to

   .\python39
   .\Scripts
   .
   # importing site will run sitecustomize.py
   import site
   

   Without these changes, you may see error

   Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
   Python runtime state: core initialized
   ModuleNotFoundError: No module named 'encodings'
   

5. Create C:\python-3.9.6-embed-amd64\sitecustomize.py with contents

   import sys
   sys.path.insert(0, '')   
   

6. Create empty directory DLLs

   PS> New-Item -type directory "C:\python-3.9.6-embed-amd64\DLLs"
   

   Directory DLLs will bypass module-loading error:

   FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\python-3.9.6-embed-amd64\\DLLs'
   

7. Check paths. From a different working directory run

   PS> C:\python-3.9.6-embed-amd64\python.exe -c "import sys; print(sys.path)"
   ['', 'C:\\python-3.9.6-embed-amd64\\python39', 'C:\\python-3.9.6-embed-amd64', 'C:\\python-3.9.6-embed-amd64\\Scripts', 'C:\\python-3.9.6-embed-amd64\\lib\\site-packages']
   

Install pip using get-pip.py

PS> Invoke-WebRequest -OutFile "C:\python-3.9.6-embed-amd64\get-pip.py" "https://bootstrap.pypa.io/get-pip.py"
PS> C:\python-3.9.6-embed-amd64\python.exe "C:\python-3.9.6-embed-amd64\get-pip.py"

Install virtualenv package

PS> C:\python-3.9.6-embed-amd64\python.exe -m pip install virtualenv

Create a new virtual environment based on the embedded distribution

From some other directory run

PS> C:\python-3.9.6-embed-amd64\python.exe -m virtualenv --copies ".venv-3.9-embedded"

This creates a new fully functioning and fully independent virtual python environment, i.e. does not use symbolic links or Windows Registry entries or $env environment variables.

Activate the virtual environment

PS> .\.venv-3.9-embedded\Scripts\activate.ps1

Verify the expected python.exe is used

PS> Get-Command python.exe | Format-Table -Autosize

CommandType Name       Version       Source
----------- ----       -------       ------
Application python.exe 3.9.6150.1013 C:\Users\user1\My-Project\.venv-3.9-embedded/Scripts\python.exe

---

Helpful References:

• Setting up python's Windows embeddable distribution (properly)
• Python 3.9.6
• Using Python on Windows - Finding Modules
• stackoverflow question using this

---

Taken from my github gist.