CODEWITHSUNDEEP
kubernetesconfigurationapache-flink
Edgar Ferney Ruiz Anzola
Edgar Ferney Ruiz Anzola

Reputation: 63

Upgrade Flink 1.10 to Flink 1.11 (Log4j on kubernetes deployment)

After upgrade Flink 1.10 to Flink 1.11, the log4j configuration is no longer working.

my previous configuration was using a library with an adapter that requires log4j 1.x and is no longer compatible with Flink 1.11

according to the new configuration, the flink-conf.yaml should look like this

log4j-console.properties: |+
    # This affects logging for both user code and Flink
    rootLogger.level = INFO
    rootLogger.appenderRef.console.ref = ConsoleAppender
    rootLogger.appenderRef.rolling.ref = RollingFileAppender

    # Uncomment this if you want to _only_ change Flink's logging
    #logger.flink.name = org.apache.flink
    #logger.flink.level = INFO

my current configuration using log4j1 looks something similar to this

log4j-console.properties: |+
         log4j.rootLogger=INFO,myappender,console
         log4j.appender.myappender=com.company.log4j.MyAppender
         log4j.appender.myappender.endpoints=http://

is there a way to tell Flink 1.11 to use log4j1 in the flink-conf.yaml file?

Upvotes: 0

Views: 724

Answers (2)

Keshav Lodhi
Keshav Lodhi

Reputation: 3182

After upgrading the Flink-1.10.2 to Flink-1.11.3 I came across the same issue in the Kubernetes and DCOS(Mesos) Flink cluster. Then to cross-verify I downloaded the Flink's binaries flink-1.11.3-bin-scala_2.12.tgz in local and tested the loggers and found it working without any change.

Flink 1.11 switched from Log4j1 to Log4j2

  • Then I have followed the steps mentioned in Flink's official documents to use Flink with Log4j1.

    1. Remove the log4j-core, log4j-slf4j-impl and log4j-1.2-api jars from the Flink's lib directory.
    2. Add the log4j, slf4j-log4j12, and log4j-to-slf4j jars to the Flink's lib directory.

  • Restarted the Kubernetes and DCOS(Mesos) Flink cluster and verified the loggers and found it working.

Upvotes: 0

Mikalai Lushchytski
Mikalai Lushchytski

Reputation: 1641

As far as I know, flink-conf.yaml does not contain log4j-console.properties section and this is a separate file. What you have specified I suppose is a part of flink-configuration-configmap.yaml cluster resource definition.

According to the flink Configuring Log4j1 Section, in order to use log4j1, you need to:

  • remove the log4j-core, log4j-slf4j-impl and log4j-1.2-api jars from the lib directory,
  • add the log4j, slf4j-log4j12 and log4j-to-slf4j jars to the lib directory,

Upvotes: 1

Related Questions