Windows Python (<=3.10.2) fails to run `python -m venv .venv`

Question

This issue has been solved, resulted in a bug report to Python.org. See the my self-answer below for the workaround until it's fixed in a future release of Python

One of my PCs got bitten by this bug which no longer allows me to create venv with the error:

Error: Command '['C:\\Users\\kesh\\test\\.venv\\Scripts\\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 101.

• This issue has been known, chronologically: v3.7.2, v3.8, v3.?, & v3.10.1
• The only known solution is to give up per-user install and use global install by checking "Install for all users" option

I'm trying to figure out what exactly is happening, but quickly running out of ideas. Here are what I've tried so far:

• On my PC, "Install for all users" works as well as per-user install on a dummy account (all using the same v3.10.2 installer). This singles out the issue to be on my Windows account. Changing the install location does not help.
• Went into venv source by running Python with venv.main(args=('.venv',)), debugging line-by-line and noted that it copies Lib\venv\scripts\nt\python.exe from the python install dir to the local .venv\Scripts folder using shutil.copyfile().
• If I run the original Lib\venv\scripts\nt\python.exe in command prompt, it runs with a message No pyvenv.cfg file (which makes sense as the .cfg file is in .venv folder which it couldn't see)
• If I call the copied .venv\Scripts\python.exe then it returns an error Unable to create process using 'C:\Users\kesh\AppData\Local\Programs\Python\Python310\python.exe' (note that the python.exe path for the process is that of the installed Python exe)
• If .venv is installed successfully (on the dummy Windows account), the above run starts a Python session as you'd expect.
• venv\scripts\nt\python.exe is different from the standard python binary and verified that this file and its source in venv\Scripts\nt are identical.
• All this points to that something in my account configuration is bothering the .venv\Scripts\python.exe to do the right thing, but my environmental variables are pretty clean and python paths are at the top of the user PATH variable.
• Currently trying to locate the source code of .venv\Scripts\python.exe but not found it yet.
• Can it be something in registry?

If you have any other ideas to try, please share.

Update #1:

• Found the source of the error message PC/launcher.c Line 814
• Possibility: CreateProcessW(NULL, cmdline,... where cmdline is the original python path in the error message, without quote. CreateProcessW documentation states executable name is deduced from the first white space–delimited token in the cmdline string. Though I replaced my actual account name with kesh it actually comprises two words and a space...

Update #2:

Solution found as posted below

Answer 1

I encountered the same problem in python3.8.10 from pyenv-win. How to solve it?

2024-05-25 12:33:04.266 [info] > "D:\Program Files\python\pyenv-win\versions\3.8.10\python.exe" ~\.vscode\extensions\ms-python.python-2024.4.1\python_files\create_venv.py --git-ignore
2024-05-25 12:33:04.266 [info] cwd: .
2024-05-25 12:33:04.495 [info] Running: D:\Program Files\python\pyenv-win\versions\3.8.10\python.exe -m venv .venv
2024-05-25 12:33:04.705 [info] Error: Command '['e:\\Python\\chatgpt-on-wechat\\.venv\\Scripts\\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 101.
2024-05-25 12:33:04.712 [info] Traceback (most recent call last):
  File "d:\Users\li\.vscode\extensions\ms-python.python-2024.4.1\python_files\create_venv.py", line 84, in run_process
2024-05-25 12:33:04.712 [info]     subprocess.run(args, cwd=os.getcwd(), check=True)
  File "D:\Program Files\python\pyenv-win\versions\3.8.10\lib\subprocess.py", line 516, in run
2024-05-25 12:33:04.712 [info]     raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['D:\\Program Files\\python\\pyenv-win\\versions\\3.8.10\\python.exe', '-m', 'venv', '.venv']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\Users\li\.vscode\extensions\ms-python.python-2024.4.1\python_files\create_venv.py", line 250, in <module>
2024-05-25 12:33:04.713 [info]     main(sys.argv[1:])
  File "d:\Users\li\.vscode\extensions\ms-python.python-2024.4.1\python_files\create_venv.py", line 219, in main
2024-05-25 12:33:04.714 [info]     run_process(
  File "d:\Users\li\.vscode\extensions\ms-python.python-2024.4.1\python_files\create_venv.py", line 86, in run_process
    raise VenvError(error_message)
__main__.VenvError: CREATE_VENV.VENV_FAILED_CREATION
2024-05-25 12:33:04.729 [error] Error while running venv creation script:  CREATE_VENV.VENV_FAILED_CREATION
2024-05-25 12:33:04.729 [error] CREATE_VENV.VENV_FAILED_CREATION
2024-05-25 12:33:05.870 [info] > .\.venv\Scripts\python.exe -I ~\.vscode\extensions\ms-python.python-2024.4.1\python_files\get_output_via_markers.py ~\.vscode\extensions\ms-python.python-2024.4.1\python_files\interpreterInfo.py
2024-05-25 12:33:05.920 [error] [Error: Command failed: e:\Python\chatgpt-on-wechat\.venv\Scripts\python.exe -I d:\Users\li\.vscode\extensions\ms-python.python-2024.4.1\python_files\get_output_via_markers.py d:\Users\li\.vscode\extensions\ms-python.python-2024.4.1\python_files\interpreterInfo.py
Unable to create process using 'D:\Program Files\python\pyenv-win\versions\3.8.10\python.exe -I d:\Users\li\.vscode\extensions\ms-python.python-2024.4.1\python_files\get_output_via_markers.py d:\Users\li\.vscode\extensions\ms-python.python-2024.4.1\python_files\interpreterInfo.py'

Answer 2

Bingo, the finding in the update #1 was the cause. The space in my username was the culprit. Although I have no idea what triggered this behavior change on my account... (anybody with an answer, please follow up.)

Let's say the per-user python is installed at

C:\Users\User Name\AppData\Local\Programs\Python\Python310

In my case, "Microsoft Visual C++ 2015-2022 Redistributable" installer (VC_redist.x64.exe) left a log file C:\Users\User (a text file with the first part of my account name as its file name). This caused python venv to use C:\Users\User as the python executable and promptly failed (see the issue tracker link below for the full explanation).

You can fix the issue in 2 ways until Python patches the problem.

Easy Fix

Simply delete the file C:\Users\User

Note: This will work until next time another installer leaves this nasty log file.

More Involved Fix

In command console, run

DIR /X C:\Users

which lists something like:

02/08/2022  11:44 AM    <DIR>                       .
02/08/2022  11:44 AM    <DIR>                       ..
11/19/2020  01:48 AM    <DIR>                       Public
02/08/2022  02:47 PM    <DIR>          USERNA~1     User Name

Open the Environmental Variables dialog and edit Path user variable's Python entries from

C:\Users\User Name\AppData\Local\Programs\Python\Python310\Scripts
C:\Users\User Name\AppData\Local\Programs\Python\Python310

to

C:\Users\USERNA~1\AppData\Local\Programs\Python\Python310\Scripts
C:\Users\USERNA~1\AppData\Local\Programs\Python\Python310

Re-open python console window or app so the path change is applied to your dev environment.

Note: This fix will work as long as you don't update python version. When you do, you need to delete the old path entries manually and update the new path entries.

Eventual Fix

I reported this bug to python bug tracker: Issue 46686. They've acknowledged the bug and have it labeled as critical with a proposed fix. So, hopefully it will get fixed in a near future release. (>3.10.2)