Filter log by matching pattern - log4j

Question

I have the following layout patter in my log4j xml file:

"%d{ISO8601} %c %p [%t] [%x] 9.5.4.RC12 %m%n"

What I want is that when ever I get a log containing message process [proc#] completed, it should be skipped. I mean every log except the one containing this message should be printed. [proc#] will contain the process number of max lenght 4. What can I desing a filter with this function in my xml config file. If so, then how?

Answer 1

ExpressionFilter can do that.

In a filter definition inside an appender definition, use an expression similar to (note LIKE is the regex match operator):

"MSG LIKE 'process \[.*\] completed'"

See ExpressionFilter javadoc here:

http://logging.apache.org/log4j/companions/apidocs/org/apache/log4j/filter/ExpressionFilter.html

Example:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/" debug="false">
    <appender name="CONSOLE">
        <param name="Target" value="System.out"/>
        <layout>
            <param name="ConversionPattern" value="%d %p [%c] - %m%n"/>
        </layout>
        <filter class="org.apache.log4j.filter.ExpressionFilter">
            <param name="expression" value="EXCEPTION ~= com.company.BackendNotAvailableException" />
            <param name="acceptOnMatch" value="false"/>
        </filter>
    </appender>
    <root>
        <priority value ="INFO" />
        <appender-ref ref="CONSOLE"/>
    </root>
</log4j:configuration>

Example taken from here: http://blog.trifork.com/2011/08/23/filtering-specific-exceptions-when-using-log4j/

Answer 2

This is how I've configured log4j to filter out by IP address. In particular 127.0.0.1 as illustrated in log4j.xml below. Note, you must also include log4j.dtd alongside log4j.xml. You must use the XML syntax.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
    <appender name="CATALINA" 
            class="org.apache.log4j.DailyRollingFileAppender">
        <param name="Append" value="true" />
        <param name="DatePattern" value="'.'yyyy-MM-dd'.log'" />
        <param name="Encoding" value="UTF-8" />
        <param name="File" value="${catalina.base}/logs/catalina" />
        <layout class="net.logstash.log4j.JSONEventLayoutV1" />
    </appender>
    <appender name="CATALINAOUT" 
            class="org.apache.log4j.ConsoleAppender">
        <param name="Append" value="true" />
        <param name="Encoding" value="UTF-8" />
        <param name="File" value="${catalina.base}/logs/catalina.out" />
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" 
                value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n" />
        </layout>


        <filter class="org.apache.log4j.filter.ExpressionFilter">
            <param name="expression" value="MSG LIKE '127.0.0.1'" />
            <param name="acceptOnMatch" value="false"/>
        </filter>


    </appender>
    <appender name="LOCALHOST" class="org.apache.log4j.DailyRollingFileAppender">
        <param name="Append" value="true" />
        <param name="DatePattern" value="'.'yyyy-MM-dd'.log'" />
        <param name="Encoding" value="UTF-8" />
        <param name="File" value="${catalina.base}/logs/localhost" />
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%d [%t] %-5p %c- %m%n" />
        </layout>
    </appender>
    <logger name="org.apache.catalina.core.ContainerBase.[Catalina].[localhost]">
        <level value="INFO" />
        <appender-ref ref="LOCALHOST" />
    </logger>
    <root>
        <level value="INFO" />
        <appender-ref ref="CATALINA" />
        <appender-ref ref="CATALINAOUT" />
    </root>
</log4j:configuration>