CODEWITHSUNDEEP
apache-kafkaapache-zookeeper
AtulyaB
AtulyaB

Reputation: 329

Kafka SASL/PLAIN Setup with SSL

I'm trying to configure SASL/PLAIN with SSL in our Kafka environment. The SSL part is done, but I'm running into the below error when starting the Zookeeper.
Has anyone configured SASL/PLAIN with SSL for Zookeeper and broker ?

Server-Broker Config

broker.id=19
port=19001

#host.name=localhost
host.name=jcia9081

#advertised.host.name=11.72.29.153

num.network.threads=3
num.io.threads=8

socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600

log.dirs=/appsdata/osp/kafka/datastore1/AIN_TEST_SSL/kafka,/appsdata/osp/kafka/datastore2/AIN_TEST_SSL/kafka,/appsdata/osp/kafka/datastore3/AIN_TEST_SSL/kafka

num.partitions=1
num.recovery.threads.per.data.dir=1

log.retention.hours=96
#log.retention.bytes=1073741824

log.segment.bytes=1073741824
log.retention.check.interval.ms=300000

log.cleaner.enable=false

replica.fetch.max.bytes=52428800

#The maximum size of a message that the server can receive.
message.max.bytes=52428800

zookeeper.connect=jcia9081:19101,jcia9131:19101,jcia9434:19101
zookeeper.connection.timeout.ms=6000
#################################
auto.create.topics.enable=false
auto.leader.rebalance.enable=true
delete.topic.enable=true
controlled.shutdown.enable=true

### SSL properties
security.inter.broker.protocol=SASL_SSL
listeners=SSL://jcia9081:19001,SASL_SSL://jcia9081:19002
ssl.keystore.location=/home/kafka/certs/jcia9081.federated.fds.jks
ssl.keystore.password=xxxx
ssl.key.password=xxxx
ssl.truststore.location=/home/kafka/certs/test.truststore.jks
ssl.truststore.password=xxx
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
#listeners=SSL://jcia9081:19001
allow.everyone.if.no.acl.found=true

Zookeeper Config

dataDir=/appsdata/osp/kafka/datastore1/AIN_TEST_SSL/zookeeper

clientPort=19101

maxClientCnxns=0

tickTime=2000

initLimit=60
syncLimit=10

server.19=jcia9081:19121:19131
server.20=jcia9131:19121:19131
server.21=jcia9434:19121:19131

JAAS Zookeeper Configuration File

Server {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret"
};

JAAS Broker Configuration File

    KafkaServer {
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="admin"
        password="admin-secret"
        user_admin="admin-secret";
        };
Client {
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="admin"
        password="admin-secret";
};

Startup Error

[2017-04-26 10:05:24,775] INFO Client environment:user.dir=/appsdata/osp/kafka/AIN_TEST_SSL/startup (org.apache.zookeeper.ZooKeeper)
[2017-04-26 10:05:24,776] INFO Initiating client connection, connectString=jcia9081:19101 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@43195e57 (org.apache.zookeeper.ZooKeeper)
[2017-04-26 10:05:24,776] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2017-04-26 10:05:24,778] DEBUG zookeeper.disableAutoWatchReset is false (org.apache.zookeeper.ClientCnxn)
[2017-04-26 10:05:24,789] DEBUG JAAS loginContext is: Client (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2017-04-26 10:05:24,795] DEBUG Awaiting connection to Zookeeper server (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:24,795] INFO Waiting for keeper state SaslAuthenticated (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:24,881] INFO successfully logged in. (org.apache.zookeeper.Login)
[2017-04-26 10:05:24,881] INFO Client will use DIGEST-MD5 as SASL mechanism. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2017-04-26 10:05:24,895] INFO Opening socket connection to server jcia9081.federated.fds/11.168.24.82:19101. Will attempt to SASL-authenticate using Login Context section 'Client' (org.apache.zookeeper.ClientCnxn)
[2017-04-26 10:05:24,968] INFO Socket connection established to jcia9081.federated.fds/11.168.24.82:19101, initiating session (org.apache.zookeeper.ClientCnxn)
[2017-04-26 10:05:24,970] DEBUG Session establishment request sent on jcia9081.federated.fds/11.168.24.82:19101 (org.apache.zookeeper.ClientCnxn)
[2017-04-26 10:05:25,001] INFO Session establishment complete on server jcia9081.federated.fds/11.168.24.82:19101, sessionid = 0x15baa8300580003, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2017-04-26 10:05:25,002] DEBUG ClientCnxn:sendSaslPacket:length=0 (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2017-04-26 10:05:25,004] DEBUG Received event: WatchedEvent state:SyncConnected type:None path:null (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:25,004] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:25,004] DEBUG Leaving process event (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:25,006] ERROR SASL authentication failed using login context 'Client'. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2017-04-26 10:05:25,006] DEBUG Received event: WatchedEvent state:AuthFailed type:None path:null (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:25,006] INFO zookeeper state changed (AuthFailed) (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:25,006] DEBUG Leaving process event (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:25,006] DEBUG Closing ZkClient... (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:25,006] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2017-04-26 10:05:25,006] DEBUG Closing ZooKeeper connected to jcia9081:19101 (org.I0Itec.zkclient.ZkConnection)
[2017-04-26 10:05:25,006] DEBUG Close called on already closed client (org.apache.zookeeper.ZooKeeper)
[2017-04-26 10:05:25,006] DEBUG Closing ZkClient...done (org.I0Itec.zkclient.ZkClient)
[2017-04-26 10:05:25,007] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure
        at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:946)
        at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:923)
        at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1230)
        at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156)
        at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130)
        at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:76)
        at kafka.utils.ZkUtils$.apply(ZkUtils.scala:58)
        at kafka.server.KafkaServer.initZk(KafkaServer.scala:327)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:200)
        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)
        at kafka.Kafka$.main(Kafka.scala:67)
        at kafka.Kafka.main(Kafka.scala)
[2017-04-26 10:05:25,009] INFO shutting down (kafka.server.KafkaServer)

Broker Start Command

    . /appsdata/osp/kafka/AIN_TEST_SSL/startup/setenv.sh
export JMX_PORT=19011
$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_CONFIG/server_AIN_TEST_SSL.properties > $LOG_DIR/kafka-broker-startup.log 2>&1 &

setenv

    export APP_HOME=/apps/osp

export JAVA_HOME=$APP_HOME/jdk1.8.0_101
export JAVA=$JAVA_HOME/bin/java

export KAFKA_HOME=$APP_HOME/kafka/v2.11-0.10.1.1
export KAFKA_CONFIG=/appsdata/osp/kafka/AIN_TEST_SSL/config

export ZK_HOME=$KAFKA_HOME
export ZK_CONFIG=$KAFKA_CONFIG

export LOG_DIR=/applogs/osp/kafka/AIN_TEST_SSL
export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$KAFKA_CONFIG/log4j.properties"

In Termninal 1

echo $KAFKA_OPTS
-Djava.security.auth.login.config=/appsdata/osp/kafka/AIN_TEST_SSL/config/jaas_AIN_TEST_SSL_zookeeper.conf

In Terminal 2

echo $KAFKA_OPTS
-Djava.security.auth.login.config=/appsdata/osp/kafka/AIN_TEST_SSL/config/jaas_AIN_TEST_SSL_server.conf

Upvotes: 3

Views: 10917

Answers (1)

Maximilien Belinga
Maximilien Belinga

Reputation: 3186

You should give the path to your jaas file by exporting KAFKA_OPTS environment variable. Copy your jaas file under /appsdata/osp/kafka/AIN_TEST_SSL/config/

I assume, this is the content of your file:

kafka_server_jaas.conf

KafkaServer {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret";
};
Client {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret";
};

zookeeper.properties

dataDir=/appsdata/osp/kafka/datastore1/AIN_TEST_SSL/zookeeper
clientPort=19101
maxClientCnxns=0
tickTime=2000
initLimit=60
syncLimit=10
server.19=jcia9081:19121:19131
server.20=jcia9131:19121:19131
server.21=jcia9434:19121:19131

#New configuration
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000

[BEGIN UPDATE]

Add the the zookeeper server jaas file and import it before starting zookeeper ensemble.

Server {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret"
};

After, export that jaas on the zookeeper terminal and start zookeeper

$ export KAFKA_OPTS="-Djava.security.auth.login.config=/appsdata/osp/kafka/AIN_TEST_SSL/config/zookeeper_jaas.conf"
$ bin/zookeeper-server-start.sh config/zookeeper.properties

Open a new terminal and hit the following commands to start your kafka server:

$ export KAFKA_OPTS="-Djava.security.auth.login.config=/appsdata/osp/kafka/AIN_TEST_SSL/config/kafka_server_jaas.conf"
$ bin/kafka-server-start.sh config/server.properties

[END UPDATE]

I think it will fix your issue.

Please look at my configurations on this post if any issue occurs:

Kafka SASL zookeeper authentication

Upvotes: 3

Related Questions