CODEWITHSUNDEEP
loggingapache-kafka
skylerl
skylerl

Reputation: 4170

Kafka Application Log Configuration

I'm trying to move the application/Log4J logs for Apache Kafka (not the data logs). This has become exceptionally hard to research as there are many log related terms with regards to Kafka. I've opened the log4j.properties, and found that the logs are written to: ${kafka.logs.dir} however, I'm not sure where to change kafka.logs.dir. We need to change this location due to low disk space on our VMs.

Any help would be much appreciated!

Upvotes: 8

Views: 15825

Answers (3)

CefBoud
CefBoud

Reputation: 224

You can either set the JVM property

-Dkafka.logs.dir=/some/path

Or the environnement variable $LOG_DIR

The former is set to the latter in kafka-run-class.sh which is called by kafka-server-start.sh

https://github.com/apache/kafka/blob/1560c5bd7e556ca6c0f49934b5ad3542ed6208fb/bin/kafka-run-class.sh#L243

Upvotes: 0

Paizo
Paizo

Reputation: 4194

One way to do it is to override the environment property LOG_DIR prior launching the kafka server, for example in a bash script:

#!/bin/sh
export LOG_DIR=/var/log/kafka
echo 'Kafka application logs set to ' $LOG_DIR
./kafka_2.11-1.1.0/bin/kafka-server-start.sh -daemon  /kafka_2.11-1.1.0/config/server.properties

You can also override the log4j.properties with your own see kafka-server-start.sh (snippet from 1.1.0):

if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
    export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
fi

Upvotes: 10

Mickael Maison
Mickael Maison

Reputation: 26865

The log4J log directory has to be passed via the following JVM property:

-Dkafka.logs.dir=/some/path

If you're using the kafka-server-start.sh to start your broker, you can just export LOG_DIR and that will automatically set the JVM property for you

Upvotes: 4

Related Questions