Kafka Topic creation - Timed out waiting for a node assignment

Question

I have deployed Single node Kafka on GCP using - https://console.cloud.google.com/marketplace/details/click-to-deploy-images/kafka

server.properties file looks like below:

Note : listeners has appeared twice. Once (commented) under "Socket Server Settings" and 2nd time (uncommented) under "Group Coordinator Settings"

############################# Socket Server Settings #############################
            
# The address the socket server listens on. If not configured, the host name will be equal to the value of
# java.net.InetAddress.getCanonicalHostName(), with PLAINTEXT listener name, and port 9092.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
#listeners=PLAINTEXT://:9092

# Listener name, hostname and port the broker will advertise to clients.
# If not set, it uses the value for "listeners".
#advertised.listeners=PLAINTEXT://your.host.name:9092
        
############################# Zookeeper #############################

# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=localhost:2181
    
############################# Group Coordinator Settings #############################

# The following configuration specifies the time, in milliseconds, that the GroupCoordinator will delay the initial consumer rebalance.
# The rebalance will be further delayed by the value of group.initial.rebalance.delay.ms as new members join the group, up to a maximum of max.poll.interval.ms.
# The default value for this is 3 seconds.
# We override this to 0 here as it makes for a better out-of-the-box experience for development and testing.
# However, in production environments the default value of 3 seconds is more suitable as this will help to avoid unnecessary, and potentially expensive, rebalances during application startup.
group.initial.rebalance.delay.ms=0
######## C2D Custom Configuration ########
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
auto.create.topics.enable=true
authorizer.class.name=kafka.security.authorizer.AclAuthorizer
allow.everyone.if.no.acl.found=true
listeners=SASL_PLAINTEXT://0.0.0.0:9092
advertised.listeners=SASL_PLAINTEXT://:9092

Question: trying to create kafka topic, but running into Error while executing topic command : Timed out waiting for a node assignment. Call: createTopics

Hence uncommented listeners and changed under section "Socket Server Settings" from #listeners=PLAINTEXT://:9092 to listeners=PLAINTEXT://localhost:9092 , But running into same issue.

Hence later commented out above as earlier and changed both listeners and advertised.listeners under "Group Coordinator Settings" as below.

listeners=SASL_PLAINTEXT://localhost:9092

advertised.listeners=SASL_PLAINTEXT://localhost:9092

Executed below commands to create topic: All of these commands are running into same issue.

kafka-topics.sh --zookeeper localhost:2181 --create --topic TestKafkaTopic --partitions 1 --replication-factor 1

kafka-topics.sh --zookeeper localhost:9092 --create --topic TestKafkaTopic --partitions 1 --replication-factor 1

kafka-topics.sh --create --bootstrap-server localhost:2181 --replication-factor 1 --partitions 1 --topic TestKafkaTopic

kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic TestKafkaTopic

kafka-topics.sh --create --bootstrap-server 127.0.0.1:9092 --replication-factor 1 --partitions 1 --topic TestKafkaTopic

Answer 1

If you want to use SASL listeners you need to provide JAAS config files to each of the CLI commands for authentication.

If that's not what you want, you should use PLAINTEXT listeners/advertised.listeners, but only one uncommented line of each

It's also not clear where you're running those commands, but the Google Cloud Console or your local machine isn't the server running Kafka - you need to use its external IP/hostname, not localhost, and --zookeeper flag has been removed in recent Kafka versions, so only use bootstrap servers

There are better options if you truly want "click to deploy" that don't require messing with config files