Problems using paho mqtt client with python 3.7

Question

I am running the following code to connect to a mqtt server.

import paho.mqtt.client as mqtt
import ssl
import uuid

client = mqtt.Client(str(uuid.uuid1()))
client.tls_set(
    "ca.crt",
    "client.crt",
    "client.key",
    cert_reqs=ssl.CERT_REQUIRED,
    tls_version=ssl.PROTOCOL_TLSv1
)
client.connect(
    "127.0.0.1",
    8883,
)
client.loop_forever()

This code works fine with python2.7 version. But when I run it with python3.7 version I am getting the below error.

Traceback (most recent call last):
  File "test.py", line 29, in <module>
    8883,
  File "virtualenvs/mqtt-xG2h6zri/lib/python3.7/site-packages/paho/mqtt/client.py", line 839, in connect
    return self.reconnect()
  File "mqtt-xG2h6zri/lib/python3.7/site-packages/paho/mqtt/client.py", line 994, in reconnect
    sock.do_handshake()
  File ".pyenv/versions/3.7.0/lib/python3.7/ssl.py", line 1108, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '127.0.0.1'. (_ssl.c:1045)

Please help me figure out how to make this work in python 3.7.

Answer 1

in Windows:

# create extfile:

echo subjectAltName=DNS:*.yourdomain.com,IP:192.168.1.104 >> ./work/extfile.cnf

# final certificate:

openssl x509 -req -sha256 -days 256 -in server_cert.csr -CA ca_cert.pem -CAkey ca_key.pem -out work/cert.pem -extfile extfile.cnf -CAcreateserial

Answer 2

Found the answer.

Actually, according to this link matching server IP address with CN field of certificate is deprecated for more than 15 years. But python versions lower than 3.7 still allow this even though it is deprecated. Therefore I had to create a certificate with the ip address of the server added in the SAN field.

Creating certificates with SAN fields is explained in this answer. But the solution in the answer uses domain names. If you are creating certificates with IP address use this command to create a certificate instead of the command in that answer.

openssl x509 -req -in server.csr \
        -extfile <(printf "subjectAltName=IP:127.0.0.1") \
        -CA ca.crt \
        -CAkey ca.key \
        -CAcreateserial -out server.crt \
        -days 365

After using these certificates the error is solved.

Answer 3

The information written here helped me a lot, after about 2 days of effort, I found a solution.

I made a script of my successful experiment. Hopefully it benefits your business.

#!/bin/bash

wd="`dirname $0`"
if [ ! -z "$wd" ]; then
    if [ $wd == "." ];then wd=`pwd`;fi  
fi

certDir="${wd}/certs";
conf_dir="/mosquitto/config" #WD

rm -rf ${certDir} 

mkdir -p ${certDir}

cd ${certDir}


#subjectAltName olmazsa self signet sertifika ile bağlanamıyor.
#Server IP - Port bilgileri

#~ IP="mqtt.eclipseprojects.io"

IP="192.168.1.10"
PORT="58883"

SUBJECT_CA="/C=TR/ST=Istanbul/L=Istanbul/O=Example/OU=CA/CN=$IP"
SUBJECT_SERVER="/C=TR/ST=Istanbul/L=Istanbul/O=Example/OU=server/CN=$IP"
SUBJECT_CLIENT="/C=TR/ST=Istanbul/L=Istanbul/O=Example/OU=client/CN=$IP"

MAX_DAYS=3650
 
SB_NAME="subjectAltName = IP:127.0.0.1" #"subjectAltName=DNS:example.com,IP:$IP"
space="################################################################################################################"

function generate_CA () {
   echo -e "$SUBJECT_CA\n$space" 
   openssl req -x509 -nodes -sha256 -newkey rsa:2048 -subj "$SUBJECT_CA" -addext  "${SB_NAME}" -days ${MAX_DAYS} -keyout ca.key -out ca.crt 
   
}

function generate_server () {
    echo -e "$SUBJECT_SERVER\n$space"

    openssl req -nodes -sha256 -new -subj "$SUBJECT_SERVER" -keyout server.key -out server.csr -addext "${SB_NAME}"

    openssl x509 -req -in server.csr \
    -extfile <(printf "${SB_NAME}") \
    -CA ca.crt \
    -CAkey ca.key \
    -CAcreateserial -out server.crt \
    -days ${MAX_DAYS}
}

function generate_client () {
   echo -e "$SUBJECT_CLIENT\n$space"
   openssl req -new -nodes -sha256 -subj "$SUBJECT_CLIENT" -out client.csr -keyout client.key 
   openssl x509 -req -sha256 -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days ${MAX_DAYS}
}

function clean () {
   echo -e "$SUBJECT_CLIENT\n$space"
    rm client.csr client.key client.crt 
    rm server.csr server.key server.crt 
    rm ca.srl ca.key ca.crt 
}


#~ clean

generate_CA
generate_server
generate_client
############################################
cd ${wd}

echo "port ${PORT}

#require_certificate true

cafile ${conf_dir}/certs/ca.crt
certfile ${conf_dir}/certs/server.crt
keyfile ${conf_dir}/certs/server.key

tls_version tlsv1.2

use_identity_as_username true
password_file ${conf_dir}/password.txt
listener 1883

persistence false
persistence_location /mosquitto/data/

" > mosquitto.conf



echo "Use Examples
#Server

mosquitto -c ${certDir}/mosquitto.conf -v

#Subcriber

mosquitto_sub -h ${IP} -p ${PORT}  --cafile ${certDir}/ca.crt --cert ${certDir}/client.crt --key ${certDir}/client.key -t temperature

#Publisher

mosquitto_pub  -h ${IP} -p ${PORT} --cafile ${certDir}/ca.crt --cert ${certDir}/client.crt --key ${certDir}/client.key -t temperature -m test_message
"

Answer 4

I have recreated the server certificate with the Common Name field as "Hostname" of the server machine (where the mosquitto broker is installed) instead of the IP address.
Basically x.x.x.x hostname
However, it works for windows

Also, setting up tls_insecure_set() as true doesn't make any sense of using a TLS certificate if anyone is using it.

Answer 5

Had a similar problem (although using a RabbitsMQ client Pika rather than MQTT), but similar error being thrown in the SSL library upon do_handshake() method in python 3.8 when trying to connect to an external endpoint rather than localhost, and I could not generate a new ssl cert as it was provided to me.

The accepted answer helped me but I think it might help others, as downgrading to python3.6 gives a much more useful error message:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/miniconda/envs/venv3.6/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 359, in __init__
    self._impl = self._create_connection(parameters, _impl_class)
  File "/root/miniconda/envs/venv3.6/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 450, in _create_connection
    raise self._reap_last_connection_workflow_error(error)
  File "/root/miniconda/envs/venv3.6/lib/python3.6/site-packages/pika/adapters/utils/io_services_utils.py", line 636, in _do_ssl_handshake
    self._sock.do_handshake()
  File "/root/miniconda/envs/venv3.6/lib/python3.6/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/root/miniconda/envs/venv3.6/lib/python3.6/ssl.py", line 694, in do_handshake
    match_hostname(self.getpeercert(), self.server_hostname)
  File "/root/miniconda/envs/venv3.6/lib/python3.6/ssl.py", line 331, in match_hostname
    % (hostname, dnsnames[0]))
ssl.CertificateError: hostname '149.176.221.21' doesn't match 'redacted.internal.url'

where redacted.internal.url is what you have to set your hostname to for the connection (in ssl context object), as that is whatever is on your certificate.

This is regardless of what ip address/url you're actually connecting to, which i this case was 149.176.221.21