CODEWITHSUNDEEP
apache-kafkaacl
Kewitschka
Kewitschka

Reputation: 1661

Kafka authorization failed only on port 9092

I use the confluent kafka docker image and have enabled authentication and authorization with the following config. KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://:9092,SASL_SSL://:9093

=> 9093 SASL_SSL
=> 9092 PLAINTEXT

Here is a part of my config:
Container environment variables

  - KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND=false
  - KAFKA_SSL_CLIENT_AUTH=required
  - KAFKA_SECURITY_INTER_BROKER_PROTOCOL=SASL_SSL
  - KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
  - KAFKA_SASL_ENABLED_MECHANISMS=PLAIN
  - KAFKA_AUTHORIZER_CLASS_NAME=kafka.security.authorizer.AclAuthorizer
  - KAFKA_SUPER_USERS="User:admin"
  - KAFKA_OPTS=-Djava.security.auth.login.config={{ kafka_secrets_dir }}/kafka_jaas.conf

kafka_jaas.conf

KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
  username="admin"
  password="admin"
  user_admin="admin"
  user_second_user="read_user";
};

Client {
org.apache.kafka.common.security.plain.PlainLoginModule required
 username="admin"
 password="admin";
};

Configure consumer ACLs

bin/kafka-acls --authorizer-properties zookeeper.connect=my.host1:2181,host2:2181,host3:2181 --add --allow-principal User:second_user --consumer --topic '*' --group '*'

Configure producer ACLs

kafka-acls --authorizer-properties zookeeper.connect=my.host1:2181,host2:2181,host3:2181 --add --allow-principal User:second_user --producer --topic '*'

I want to use kafka over both ports. 9093 with SSL encryption and 9092 witout. Therefore I tested it with a simple console consumer/producer. Port 9093 works fine, I can consume and produce messages. The problem is that it does not work over port 9092. I always get an authentication error TopicAuthorizationException: Not authorized to access topics: [test_topic]. I tested it with the "second_user" and even with the super user "admin". Why does it only work with the secured port? Did I miss any config?

Console consume via port 9093 (working)

#consumer.properties

ssl.endpoint.identification.algorithm=
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
ssl.truststore.location=/home/vagrant/kafka-2.8.0/ssl/kafka.truststore.jks
ssl.truststore.password=changeme
ssl.protocol=TLS
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
    username="admin" \
    password="admin";


# create consumer => This is working!
/bin/kafka-console-consumer.sh --bootstrap-server host1:9093,host2:9093,host3:9093 --topic test_topic --from-beginning --consumer.config consumer.properties

Console consume via port 9092 (not working)

#consumer.properties
security.protocol=PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
    username="admin" \
    password="admin";

#create consumer
kafka-console-consumer.sh --bootstrap-server host1:9092,host2:9092,host3:9092 --topic test_topic --from-beginning --consumer.config consumer.properties

=>TopicAuthorizationException: Not authorized to access topics: [test_topic]

I also tested it with python and the confluent-kafka-python package(not working).
test.py

self.consumer = Consumer({
            'bootstrap.servers': "host1:9092,host2:9092,host3:9092",
            'group.id': f"test",
            'security.protocol': "PLAINTEXT",
            'sasl.mechanism': 'PLAIN',
            'sasl.username': 'admin',
            'sasl.password': "admin"
        })
=> FindCoordinator response error: Group authorization failed

Upvotes: 0

Views: 3666

Answers (1)

Ran Lupovich
Ran Lupovich

Reputation: 1831

You did not enable authentication on port 9092,

with combination of KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND

, you are getting authorization failure,

to fix it you should change to SASL_PLAINTEXT to allow SASL authentication without TLS encryption

PLAINTEXT://:9092 -> SASL_PLAINTEXT://:9092

Upvotes: 1

Related Questions