CODEWITHSUNDEEP
apache-kafkaapache-zookeepersasldigest-authenticationsasl-scram
FoxxDE
FoxxDE

Reputation: 1

Apache Kafka SASL_SSL configure failure ( ERROR SASL authentication failed using login context 'Client'.)

I want to set up Kafka with SASL_SSL in a docker enviroment kafka should be albe to recives message encrypted over the puplic internet in addition, telegraf grafana and more are used in the backend everything runs flawlessly

only the zookeeper and kafka should be allowed to communicate on the internet

kafka should be encrypted with SASL_SSL SCRAM-SHA-512
the connection between kafka and zookeeper should run via DIGEST-MD5

but i can't find a solution because i always get the following errors

ERROR SASL authentication failed using login context 'Client'. (org.apache.zookeeper.client.ZooKeeperSaslClient

** ERROR Error occurred while connecting to Zookeeper server[zookeeper:2181,zookeeper:2182]. Authentication failed. **

** javax.security.sasl.SaslException: Error in authenticating with a Zookeeper Quorum member: the quorum member's saslToken is null.**

i also add the full log

===> Configuring ...
Running in Zookeeper mode...
SSL is enabled.
SASL is enabled.
===> Running preflight checks ... 
===> Check if /var/lib/kafka/data is writable ...
===> Check if Zookeeper is healthy ...
[2023-08-14 14:15:47,648] INFO SASL is enabled. java.security.auth.login.config=/etc/kafka/sasl.jaas.config (io.confluent.admin.utils.ClusterStatus)
[2023-08-14 14:15:47,677] INFO Client environment:zookeeper.version=3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a, built on 04/08/2021 16:35 GMT (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,677] INFO Client environment:host.name=353d245d9a35 (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,677] INFO Client environment:java.version=11.0.18 (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,677] INFO Client environment:java.vendor=Azul Systems, Inc. (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,678] INFO Client environment:java.home=/usr/lib/jvm/zulu11-ca (org.apache.zookeeper.ZooKeeper)
     
8-14 14:15:47,678] INFO Client environment:java.class.path=/usr/share/java/cp-base-new/zookeeper-jute-3.6.3.jar:/usr/share/java/cp-base-new/jackson-dataformat-yaml-2.14.2.jar:/usr/share/java/cp-base-new/argparse4j-0.7.0.jar:/usr/share/java/cp-base-new/commons-cli-1.4.jar:/usr/share/java/cp-base-new/metrics-core-2.2.0.jar:/usr/share/java/cp-base-new/kafka-storage-7.4.1-ccs.jar:/usr/share/java/cp-base-new/jackson-databind-2.14.2.jar:/usr/share/java/cp-base-new/jackson-annotations-2.14.2.jar:/usr/share/java/cp-base-new/disk-usage-agent-7.4.1.jar:/usr/share/java/cp-base-new/scala-reflect-2.13.10.jar:/usr/share/java/cp-base-new/kafka-metadata-7.4.1-ccs.jar:/usr/share/java/cp-base-new/lz4-java-1.8.0.jar:/usr/share/java/cp-base-new/json-simple-1.1.1.jar:/usr/share/java/cp-base-new/re2j-1.6.jar:/usr/share/java/cp-base-new/snakeyaml-2.0.jar:/usr/share/java/cp-base-new/metrics-core-4.1.12.1.jar:/usr/share/java/cp-base-new/gson-2.9.0.jar:/usr/share/java/cp-base-new/slf4j-api-1.7.36.jar:/usr/share/java/cp-base-new/scala-collection-compat_2.13-2.10.0.jar:/usr/share/java/cp-base-new/kafka-group-coordinator-7.4.1-ccs.jar:/usr/share/java/cp-base-new/paranamer-2.8.jar:/usr/share/java/cp-base-new/audience-annotations-0.5.0.jar:/usr/share/java/cp-base-new/slf4j-reload4j-1.7.36.jar:/usr/share/java/cp-base-new/zstd-jni-1.5.2-1.jar:/usr/share/java/cp-base-new/jackson-dataformat-csv-2.14.2.jar:/usr/share/java/cp-base-new/jose4j-0.9.3.jar:/usr/share/java/cp-base-new/jmx_prometheus_javaagent-0.18.0.jar:/usr/share/java/cp-base-new/common-utils-7.4.1.jar:/usr/share/java/cp-base-new/kafka_2.13-7.4.1-ccs.jar:/usr/share/java/cp-base-new/kafka-clients-7.4.1-ccs.jar:/usr/share/java/cp-base-new/snappy-java-1.1.10.1.jar:/usr/share/java/cp-base-new/jopt-simple-5.0.4.jar:/usr/share/java/cp-base-new/zookeeper-3.6.3.jar:/usr/share/java/cp-base-new/scala-logging_2.13-3.9.4.jar:/usr/share/java/cp-base-new/scala-java8-compat_2.13-1.0.2.jar:/usr/share/java/cp-base-new/jackson-core-2.14.2.jar:/usr/share/java/cp-base-new/jolokia-jvm-1.7.1.jar:/usr/share/java/cp-base-new/logredactor-1.0.12.jar:/usr/share/java/cp-base-new/kafka-server-common-7.4.1-ccs.jar:/usr/share/java/cp-base-new/kafka-storage-api-7.4.1-ccs.jar:/usr/share/java/cp-base-new/jackson-datatype-jdk8-2.14.2.jar:/usr/share/java/cp-base-new/kafka-raft-7.4.1-ccs.jar:/usr/share/java/cp-base-new/scala-library-2.13.10.jar:/usr/share/java/cp-base-new/jackson-module-scala_2.13-2.14.2.jar:/usr/share/java/cp-base-new/jolokia-core-1.7.1.jar:/usr/share/java/cp-base-new/utility-belt-7.4.1.jar:/usr/share/java/cp-base-new/logredactor-metrics-1.0.12.jar:/usr/share/java/cp-base-new/reload4j-1.2.19.jar:/usr/share/java/cp-base-new/minimal-json-0.9.5.jar (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,678] INFO Client environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,679] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,679] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,679] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,679] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,680] INFO Client environment:os.version=5.10.0-21-amd64 (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,680] INFO Client environment:user.name=appuser (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,680] INFO Client environment:user.home=/home/appuser (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,680] INFO Client environment:user.dir=/home/appuser (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,681] INFO Client environment:os.memory.free=55MB (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,681] INFO Client environment:os.memory.max=984MB (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,681] INFO Client environment:os.memory.total=62MB (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,687] INFO Initiating client connection, connectString=zookeeper:2181,zookeeper:2182 sessionTimeout=40000 watcher=io.confluent.admin.utils.ZookeeperConnectionWatcher@221af3c0 (org.apache.zookeeper.ZooKeeper)
[2023-08-14 14:15:47,694] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-08-14 14:15:47,704] INFO jute.maxbuffer value is 1048575 Bytes (org.apache.zookeeper.ClientCnxnSocket)
[2023-08-14 14:15:47,715] INFO zookeeper.request.timeout value is 0. feature enabled=false (org.apache.zookeeper.ClientCnxn)
[2023-08-14 14:15:47,837] INFO Client successfully logged in. (org.apache.zookeeper.Login)
[2023-08-14 14:15:47,842] INFO Client will use DIGEST-MD5 as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2023-08-14 14:15:47,884] INFO Opening socket connection to server zookeeper/192.168.112.2:2182. (org.apache.zookeeper.ClientCnxn)
[2023-08-14 14:15:47,884] INFO SASL config status: Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
[2023-08-14 14:15:47,899] INFO Socket connection established, initiating session, client: /192.168.112.3:47736, server: zookeeper/192.168.112.2:2182 (org.apache.zookeeper.ClientCnxn)
[2023-08-14 14:15:47,905] WARN Session 0x0 for sever zookeeper/192.168.112.2:2182, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
EndOfStreamException: Unable to read additional data from server sessionid 0x0, likely server has closed socket
    at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
[2023-08-14 14:15:48,808] INFO Client successfully logged in. (org.apache.zookeeper.Login)
[2023-08-14 14:15:48,809] INFO Client will use DIGEST-MD5 as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2023-08-14 14:15:48,810] INFO Opening socket connection to server zookeeper/192.168.112.2:2181. (org.apache.zookeeper.ClientCnxn)
[2023-08-14 14:15:48,811] INFO SASL config status: Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
[2023-08-14 14:15:48,815] INFO Socket connection established, initiating session, client: /192.168.112.3:58200, server: zookeeper/192.168.112.2:2181 (org.apache.zookeeper.ClientCnxn)
[2023-08-14 14:15:48,823] INFO Session establishment complete on server zookeeper/192.168.112.2:2181, session id = 0x101527ddfc400fd, negotiated timeout = 40000 (org.apache.zookeeper.ClientCnxn)
[2023-08-14 14:15:48,842] ERROR SASL authentication failed using login context 'Client'. (org.apache.zookeeper.client.ZooKeeperSaslClient)
javax.security.sasl.SaslException: Error in authenticating with a Zookeeper Quorum member: the quorum member's saslToken is null.
    at org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslToken(ZooKeeperSaslClient.java:310)
    at org.apache.zookeeper.client.ZooKeeperSaslClient.respondToServer(ZooKeeperSaslClient.java:270)
    at org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:936)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:98)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
[2023-08-14 14:15:48,843] ERROR Error occurred while connecting to Zookeeper server[zookeeper:2181,zookeeper:2182]. Authentication failed.  (io.confluent.admin.utils.ClusterStatus)
[2023-08-14 14:15:48,843] INFO EventThread shut down for session: 0x101527ddfc400fd (org.apache.zookeeper.ClientCnxn)
Using log4j config /etc/kafka/log4j.properties

Here are my configs

server.properties

# Broker Basics
broker.id=1
listeners=SASL_PLAINTEXT://kafka:9092,SSL://kafka:9093,SASL_SSL://kafka:9094
num.network.threads=3
num.io.threads=8
zookeeper.connect=kafka:2181,kafka:2182
zookeeper.set.acl=true
authorizer.class.name=kafka.security.auth.AclAuthorizer
auto.create.topics.enable=false
num.partitions=1
default.replication.factor=1
socket.request.max.bytes=1000000000
max.request.size=1000000000
security.inter.broker.protocol=SSL
ssl.client.auth=required
ssl.endpoint.identification.algorithm=
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512
sasl.enabled.mechanisms=SCRAM-SHA-512,DIGEST-MD5
##CA##
ssl.truststore.location=/etc/kafka/truststore/kafka.truststore.jks
ssl.truststore.password=password
ssl.keystore.location=/etc/kafka/keystore/kafka.keystore.jks
ssl.keystore.password=password
ssl.key.password=password
sasl.enabled.mechanisms=PLAIN

Kafka Jaas

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="secret";

    org.apache.zookeeper.server.auth.DigestLoginModule required
    username="admin"
    password="secret";
};

Client {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="admin"
    password="secret";
};

Btw tryed mamy combinnation like just Server{ } as name and more

zookeeper.properties

dataDir=/var/lib/zookeeper/data
dataLogDir=/var/lib/zookeeper/log

clientPort=2181
tickTime=2000
maxClientCnxns=100
autopurge.snapRetainCount=12
autopurge.purgeInterval=168
initLimit=10
syncLimit=5
secureClientPort=2182
ssl.keyStore.location=/etc/kafka/keystore/zookeeper.keystore.jks
ssl.keyStore.password=password
ssl.trustStore.location=/etc/kafka/truststore/zookeeper.truststore.jks
ssl.trustStore.password=password
serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
authProvider=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
admin.enableServer=false

Zookeeper.jaas

Server {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    user_admin="secret";
};

and thats my docker Compose:

version: '3.6'
services:
  zookeeper:
    image: 'confluentinc/cp-zookeeper:latest'
    container_name: zookeeper
    restart: always
    ports:
      - '2181:2181'
      - '2182:2182'
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_SECURE_CLIENT_PORT: 2182
      serverCnxnFactory: org.apache.zookeeper.server.NettyServerCnxnFactory
      ZOOKEEPER_OPTS: -Djava.security.auth.login.config=/etc/kafka/zookeeper.jaas
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 10
      ZOOKEEPER_SYNC_LIMIT: 5
      ZOOKEEPER_DATADIR_AUTOCREATE: "false"
      ZOOKEEPER_MAX_CLIENT_CNXNS: 100
      ZOOKEEPER_AUTOPURGE_SNAP_RETAIN_COUNT: 12
      ZOOKEEPER_AUTOPURGE_PURGE_INTERVAL: 168
      ZOOKEEPER_ADMIN_ENABLE_SERVER: "false"
      ZOOKEEPER_AUTH_PROVIDER: org.apache.zookeeper.server.auth.SASLAuthenticationProvider
      ZOOKEEPER_REQUIRE_CLIENT_AUTH_SCHEME: sasl
      ZOOKEEPER_JAAS_LOGIN_RENEW: 3600000
      ZOOKEEPER_AUTH_PROVIDER_X509: org.apache.zookeeper.server.auth.X509AuthenticationProvider
      ZOOKEEPER_SERVER_CNXN_FACTORY: org.apache.zookeeper.server.NettyServerCnxnFactory
      ZOOKEEPER_SSL_PROTOCOL: TLSv1.2
      ZOOKEEPER_SSL_TRUSTSTORE_LOCATION: /etc/kafka/truststore/zookeeper.truststore.jks
      ZOOKEEPER_SSL_KEYSTORE_LOCATION: /etc/kafka/keystore/zookeeper.keystore.jks
      ZOOKEEPER_SSL_KEYSTORE_PASSWORD: password
      ZOOKEEPER_SSL_TRUSTSTORE_PASSWORD: password
      ZOOKEEPER_DIGEST_AUTHENTICATION_PROVIDER_SUPERDIGEST: admin:sha1hashpassword
      KAFKA_LOG4J_ROOTLOGLEVEL: DEBUG

    volumes:
      - /data/zookeeper/zookeeper.properties:/etc/kafka/zookeeper.properties
      - /data/zookeeper/zookeeper.jaas:/etc/kafka/zookeeper.jaas
      - /data/zookeeper/truststore:/etc/kafka/truststore
      - /data/zookeeper/keystore:/etc/kafka/keystore
    networks:
      - kafka_network
 kafka:
    image: 'confluentinc/cp-kafka:latest'
    container_name: kafka
    restart: always
    ports:
      - '9093:9093'
      - '9094:9094'
    depends_on:
      - zookeeper
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181,zookeeper:2182
      KAFKA_LISTENERS: SSL://kafka:9093,SASL_SSL://kakfa:9094
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,SASL_SSL://kafka:9093
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_SSL_KEYSTORE_FILENAME: kafka.keystore.jks
      KAFKA_SSL_KEY_CREDENTIALS: password.key
      KAFKA_SSL_KEYSTORE_CREDENTIALS: password.key
      KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/sasl.jaas.config"
    volumes:
      - /data/kafka/server.properties:/etc/kafka/server.properties
      - /data/kafka/keystore/kafka.keystore.jks:/etc/kafka/secrets/kafka.keystore.jks
      - /data/kafka/truststore/kafka.truststore.jks:/etc/kafka/secrets/truststore.keystore.jks
      - /data/kafka/password.key:/etc/kafka/secrets/password.key
      - /data/kafka/sasl.jaas.config/etc/kafka/sasl.jaas.config
    networks:
      - kafka_network


networks:
  kafka_network:

if you need any more details feel free to ask i can provide more

it would help me a lot if someone know a good dokumentation to build this in a docker or if someone know what i do wrong

Upvotes: 0

Views: 2702

Answers (1)

FoxxDE
FoxxDE

Reputation: 1

I solved my Problem my self now.

if you have the same problem

highly recommand to switch to Apache KRaft instead of Zookeeper becouse Zookeeper is outdated.

Also dont use self sign cerficates this would work out. use a lets encrypt cerficate or buy cerficate for the usecase

also make sure that the server can read the data with the sasl passwords

Upvotes: 0

Related Questions