Reputation: 1696
I'm trying to build Python and OpenSSL from source in a container. Both seem to build correctly, but Python does not successfully create the _ssl
module.
I've found a few guides online that say to un-comment and lines from Python-3.X.X/Modules/Setup
and add the --openssldir=/usr/local/ssl
flag to the ./configure
step for OpenSSL. I do these in my dockerfile. This has had the effect that, during the ./configure
output for Python, I see the following line.
checking for X509_VERIFY_PARAM_set1_host in libssl... yes
Yet I receive the following errors:
[91m*** WARNING: renaming "_ssl" since importing it failed: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by build/lib.linux-x86_64-3.8/_ssl.cpython-38-x86_64-linux-gnu.so)
[0m[91m*** WARNING: renaming "_hashlib" since importing it failed: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_1' not found (required by build/lib.linux-x86_64-3.8/_hashlib.cpython-38-x86_64-linux-gnu.so)
[0m
Python build finished successfully!
...
Following modules built successfully but were removed because they could not be imported:
_hashlib _ssl
Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381
If ./configure
finds X509...
, why am I still getting the hashlib and ssl errors?
The full Dockerfile, FWIW:
FROM jenkins/jenkins:lts
USER root
RUN apt-get update && apt-get install -y apt-utils gcc make zlib1g-dev \
build-essential libffi-dev checkinstall libsqlite3-dev
RUN wget https://www.openssl.org/source/openssl-1.1.1d.tar.gz && \
tar xzf openssl-1.1.1d.tar.gz && \
cd openssl-1.1.1d && \
./config -Wl,--enable-new-dtags,-rpath,'$(LIBRPATH)' --prefix=/usr/local/ssl --openssldir=/usr/local/ssl && \
make && \
make test && \
make install
RUN wget -q https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tgz && \
tar -xzf Python-3.8.2.tgz && \
cd Python-3.8.2 && \
./configure && \
make && \
make install
USER jenkins
Upvotes: 7
Views: 28653
Reputation: 23591
The following worked for me on Amazon's EC2, with the default CentOS 7 system.
First, the openssl libraries on CentOS 7 are too old (Python 3.9+ wants openssl 1.1.1+ and the version available is 1.0.x). Install the newer ones:
sudo yum install openssl11-devel
Note: since writing this answer, Amazon has end-of-life'd the openssl11-devel package and updated openssl-devel to 3.0.8. 3.0.8 is more than enough for Python, so now you can just do yum install openssl-devel
.
Unfortunately, CentOS doesn't actually put the SSL libraries anywhere that Python can find them. With some trial and error, I found that this makes ./configure
happy:
export OPENSSL_LIBS=/usr/lib64/libssl.so
./configure \
--with-openssl=/usr \
--with-openssl-rpath=/usr/lib64 \
--enable-optimizations
Explanation:
--with-openssl
takes a path that ./configure appends include/openssl/ssl.h
to, so make sure that is there for your system!When you run ./configure
, you're looking for a line near the end of the output like:
checking for stdlib extension module _ssl... yes
If you see missing
instead of yes
, search config.log for openssl, which should give you some guidance about where it's screwing up.
Hopefully this saves someone else the many hours I spent figuring this out.
Upvotes: 15
Reputation: 4699
I had the same problem and after 3+ hours of searching THIS is what actually worked:
Error: "ssl module is not available" when installing package with pip3
———————————————————-
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by ‘SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)’: /simple/pip/
The fix:
———————————————————-
sudo yum install openssl11 openssl11-devel
mkdir /usr/local/openssl11
cd /usr/local/openssl11
ln -s /usr/lib64/openssl11 lib
ln -s /usr/include/openssl11 include
The above code creates an alternate path for the latest openssl11
and symlinks it in a place that matches the folder structure that python expects.
Then proceed with the original steps in the guide (https://linuxstans.com/how-to-install-python-centos/)
and ./configure like this within your python install folder:
./configure -–enable-optimizations -–with-openssl=/usr/local/openssl11
make altinstall
the altinstall
is important so that you don't overwrite the system default python. You'll have to invoke it as python3.10
from command line thereafter.
I tested this on CentOS 7 and Python-3.10.8.
Upvotes: 7
Reputation: 876
Following modules built successfully but were removed because they could not be imported: _hashlib _ssl Could not build the ssl module! Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host(). LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381
It seems like installation issue when building openssl from source. For build failure on _ssl
module, try extra options like --with-openssl
, CFLAGS
and LDFLAGS
when configuring Python using the script ./configure
, e.g.
./configure --with-openssl=/PATH/TO/YOUR/OPENSSL_INSTALL_FOLDER/ \
--enable-optimizations \
--with-ssl-default-suites=openssl \
CFLAGS="-I/PATH/TO/YOUR/OPENSSL_INSTALL_FODLER/include" \
LDFLAGS="-L/PATH/TO/YOUR/OPENSSL_INSTALL_FODLER/"
Also try this command openssl version
, if it reports error like this :
/usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found
that means there is linking problem on your openssl library, I'm not sure if you're on Linux or other system, but for Linux system, you can manually modify the links to openssl library to fix the problem as described in my answer at here.
Reference
Building Python 3.7.1 - SSL module failed
Python 3.7.0 wont compile with SSL Support 1.1.0
Upvotes: 7
Reputation: 498
I reckon that Jenkins Image comes with some openssl version installed that is not 1.1.1, hence you find X509... in libssl but cant build.
Regarding said config option, you can spin up the container with bash as CMD, copy the config from within the container to the machine where the Image lies, edit ist and bake your version of the config into the Image.
Upvotes: 1