CODEWITHSUNDEEP
logstashapache-kafka
Gaurav J
Gaurav J

Reputation: 76

Logstash 5 is not outputting filebeat input and Kafka input

Below is configuration on Logstash 5 server

input {
kafka {
topics => ["logstash_logs"]
bootstrap_servers => "zk_server:2181"
codec => plain {
format => "%{message}"
}
}
}

input {
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}

output{
stdout{}
}

filebeat is configured on same machine as that of Logstash server. It is configured correctly as well.

Still I cannot see data in /var/log/logstash/logstash.stdout .

logstash is started with -log.level=debug still I cannot see any thing in log /var/log/logstash/logstash.log

Please help me to debug setup

Upvotes: 0

Views: 725

Answers (2)

Gman
Gman

Reputation: 2453

Change the port number from 2181 to 9092, had the same problem. Check the logs, do a tail on the logstash log tail -f -10 /var/log/logstash/logstash.log you should see ipaddress:2181 connection refused.

Upvotes: 0

dcode
dcode

Reputation: 33

bootstrap_servers refers to the Kafka broker itself, not the Zookeeper instance. Starting with Kafka 0.9, the consumer doesn't directly deal with Zookeeper anymore (unless it really wants to).

The real problem here is that Logstash doesn't provide enough information for a user to deduce why this isn't working. Also the documentation doesn't make it clear that this is Kafka and not Zookeeper. They do provide a hint in the default port number of :9092, however.

See https://www.elastic.co/guide/en/logstash/5.0/plugins-inputs-kafka.html#plugins-inputs-kafka-bootstrap_servers

Upvotes: 2

Related Questions