Install Python 3.10 or 3.11 with OpenSSL on MAC Apple Silicon

Question

I have tried multiple times to get Python working on my Mac M2 and always have trouble openssl.

I have used both Stack overflow questions (which are out of date) and ChatGPT to guide me on installation and still cannot get a working environment.

I have tried with pyenv, asdf and know that it is happening on the python compilation, not as part of the version managers.

My goal is to write software using python 3.11.x and 3.10.x, the reason for 3.10 is that some of the software I wanted to use (WhisperAI) will not work on 3.11 so I will use virtual environments.

I am going to list every command I run in this post and show where it fails.

remove any remnants of python

sudo rm -rf /Library/Frameworks/Python.framework/Versions/3.10
sudo rm -rf /usr/local/bin/python*
sudo rm -rf /usr/local/bin/pip*
sudo rm -rf /usr/local/lib/python*
sudo rm -rf /usr/local/include/python*

sudo rm -rf /Library/Frameworks/Python.framework
sudo rm -rf ~/Library/Caches/pip
sudo rm -rf /Users/<username>/.local/lib/python*
sudo rm -rf /Users/<username>/.cache/pip

brew uninstall python

# done

which python3
# => /usr/bin/python3

python --version
# => Python 3.9.6

Install Python

brew install python@3.10
brew install python@3.11

python3 --version
Python 3.11.3

Setup pyenv

brew install pyenv

echo 'if command -v pyenv 1>/dev/null 2>&1; then eval "$(pyenv init -)"; fi' >> ~/.zshrc

source ~/.zshrc

Find out what version of 3.10 and 3.11 to install

pyenv install --list | grep 3.10
pyenv install --list | grep 3.11

# There is a 3.10.11 and 3.11.3

Use pyenv to install latest versions

This is where the first issues with SSL start happening

I get exactly the same issue with 3.10.11 and 3.11.3

pyenv install 3.11.3
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Installing Python-3.11.3...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/xxxxxxxxxx/.pyenv/versions/3.11.3/lib/python3.11/ssl.py", line 100, in <module>
    import _ssl             # if we can't import it, let the error propagate
    ^^^^^^^^^^^
ModuleNotFoundError: No module named '_ssl'
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?

Please consult to the Wiki page to fix the problem.
https://github.com/pyenv/pyenv/wiki/Common-build-problems


BUILD FAILED (OS X 13.3.1 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/98/545gr7v16mscl5dhgtymzw5r0000gn/T/python-build.20230530123200.52419
Results logged to /var/folders/98/545gr7v16mscl5dhgtymzw5r0000gn/T/python-build.20230530123200.52419.log

Last 10 log lines:
  File "/private/var/folders/98/545gr7v16mscl5dhgtymzw5r0000gn/T/python-build.20230530123200.52419/Python-3.11.3/Lib/hashlib.py", line 123, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type blake2s
Looking in links: /var/folders/98/545gr7v16mscl5dhgtymzw5r0000gn/T/tmpij1ox7r9
Processing /private/var/folders/98/545gr7v16mscl5dhgtymzw5r0000gn/T/tmpij1ox7r9/setuptools-65.5.0-py3-none-any.whl
Processing /private/var/folders/98/545gr7v16mscl5dhgtymzw5r0000gn/T/tmpij1ox7r9/pip-22.3.1-py3-none-any.whl
Installing collected packages: setuptools, pip
  WARNING: The scripts pip3 and pip3.11 are installed in '/Users/xxxxxxxxxx/.pyenv/versions/3.11.3/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.3.1 setuptools-65.5.0

ASDF has the same problem as PyEnv

asdf install python 3.11.3

Check what version of openssl is installed

brew list --versions openssl

openssl@3 3.1.0

# also installed 1.1, but it does not show up when you list versions
brew install openssl@1.1

Testing with openssl@1.1 and openssl@3

I have attempted setting flags during compilation.

# Attempt #1
ENV | grep FLAGS
LDFLAGS=-L/usr/local/opt/openssl@1.1/lib
CPPFLAGS=-I/usr/local/opt/openssl@1.1/include

# Attempt #2
ENV | grep FLAGS
LDFLAGS=-L/usr/local/opt/openssl@3/lib
CPPFLAGS=-I/usr/local/opt/openssl@3/include

Using openssl@1.1

Using openssl@3

Maybe the issue is ARM64 vs AMD64?

For a while, I thought the issue was related to an incompatible Homebrew installation.

My Mac M2 (apple silicon) was initially installed 6 months ago using the Mac Migration Assistant from an 16 inch MacBook Pro (Intel).

The Mac Min is on ARM64 architecture. The MacBook was on AMD64 architecture.

When Homebrew transferred over, it kept working correctly, but maybe it was configured for a different architecture when it comes to compiling libraries.

I have reinstalled Homebrew, but I still cannot get PyEnv to install.

# uninstall old (Intel Version)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall.sh)"

# removed reference to brew on my computer
sudo rm -rf /usr/local/bin/brew

# closed down my terminal and then started a new terminal session

Re-install Homebrew

# install (Apple Version)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Reinstall openssl@1.1
brew install openssl@1.1

I also cleaned up some files that were suggested by 'brew doctor'

Answer 1

I was not able to get python compilation working on Apple Silicon using ASDF or PyEnv, but I was able to get multi-version python working using anaconda

From what I can tell, a precompiled version of python is downloaded so you don't need to got through the compilation process.

conda create -n py11 python=3.11.3
conda create -n py10 python=3.10.11

conda activate py11
conda activate py10