CODEWITHSUNDEEP
javalogginglog4j
Martinecko
Martinecko

Reputation: 1949

Log4j - missing logs in zipped files under heavy load

We are encountering problem with log4j logging under heavy logging. We periodically miss some logs (~4-20 minutes each 1 hour) in files, that are rolled out and zipped.

Configuration:

<appender name="MY_LOG" class="org.apache.log4j.appender.TimeAndSizeRollingAppender">
    <param name="File"   value="../logs/mylog.log" />
    <param name="DatePattern" value=".yyyy-MM-dd"/>
    <param name="MaxFileSize" value="40MB"/>
    <param name="MaxRollFileCount" value="100"/>
    <param name="ScavengeInterval" value="300000"/>
    <param name="encoding" value="UTF-8" />
    <param name="CompressionAlgorithm" value="ZIP"/>
    <param name="CompressionMinQueueSize" value="5"/>
    <param name="CompressionThreadPriority" value="1"/>
    <param name="CompressionLevel" value="1"/>
    <param name="CompressionUseBlockingQueue" value="true"/>
    <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="%d %-5p %22.22t| %22.22c{1}| %m%n"/>
    </layout>
</appender>

Property <param name="CompressionUseBlockingQueue" value="true"/> was added as attempt to prevent logs from getting lost. We thought (and we still do) that logs get lost while files (when there is 5 temp files) are merged and zipped. That they might get overwritten somehow. However, this property didn't work.

I'm not sure where to look for problem, or how to change configuration, so logs wouldn't get lost, or overwritten.

Any suggestions what we might try?

Thanks

Upvotes: 0

Views: 1248

Answers (1)

Gyanendra Dwivedi
Gyanendra Dwivedi

Reputation: 5547

You could try to create a copy of the log to some temp location (or at the same location, if space is not a constraint); and then zip it.

It might happen that the point of time zipping is happening, the logger buffer is trying to wrte back and faling. Since log4j fails silently, it is difficult to trace such error.

Upvotes: 1

Related Questions