CODEWITHSUNDEEP
loggingpython-3.x
Bernmeister
Bernmeister

Reputation: 255

Limit Python log file

I'm using Python3 (Ubuntu 14.04) to write out a log file. I want to limit the size of the log file and so chose RotatingFileHandler. Additionally, I don't want to rotate the log files (.log becomes .log.1 and so on) - I just want to limit the size of the log file.

I've tried using RotatingFileHandler like this:

filehandler = logging.handlers.RotatingFileHandler( "app.log", "a", 1000, 0, None, True )
logging.basicConfig( format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s", level = logging.DEBUG, handlers = [ filehandler ] )

What I'm finding is the log file continues to grow and is never truncated or "wrapped" at the limit.

I would have thought setting backupCount = 0 would cause the log file size to be checked before writing and if there's room for the log message, write it, otherwise empty the file and write out the message.

Any ideas?

Thanks in advance,

Bernmeister.

Upvotes: 4

Views: 5453

Answers (2)

Bernmeister
Bernmeister

Reputation: 255

Inspired by logc's answer, I took a more brutal approach: I just delete the log file.

class TruncatedFileHandler( logging.handlers.RotatingFileHandler ):
    def __init__( self, filename, mode = "a", maxBytes = 0, encoding = None, delay = 0 ):
        super( TruncatedFileHandler, self ).__init__( filename, mode, maxBytes, 0, encoding, delay )


    def doRollover( self ):
        if self.stream: self.stream.close()

        if os.path.exists( self.baseFilename ): os.remove( self.baseFilename )

        self.mode = "a" # Not sure why but "w" behaves in the same way as "a".
        self.stream = self._open()

One point I've found with my solution and that of logc's is once doRollover() is called, setting self.mode="a" or self.mode="w" makes no difference in terms of writing to the file. That is, I expected "w" to start writing from the beginning of the file EACH time but it does not (it behaves in the same way as "a" and writes from the last place it wrote).

Upvotes: 1

logc
logc

Reputation: 3923

If you have a look at the source code of the RotatingFileHandler, you will see that a backupCount of 0 just closes and then reopens the log's stream. Here is a modified version that does what you want, discarding the old file:

import os
import logging
import logging.handlers


class TruncatedFileHandler(logging.handlers.RotatingFileHandler):
    def __init__(self, filename, mode='a', maxBytes=0, encoding=None, delay=0):
        super(TruncatedFileHandler, self).__init__(
            filename, mode, maxBytes, 0, encoding, delay)

    def doRollover(self):
        """Truncate the file"""
        if self.stream:
            self.stream.close()
        dfn = self.baseFilename + ".1"
        if os.path.exists(dfn):
            os.remove(dfn)
        os.rename(self.baseFilename, dfn)
        os.remove(dfn)
        self.mode = 'w'
        self.stream = self._open()

Here is how to use it:

filehandler = TruncatedFileHandler("app.log", "w", 1000)
logging.basicConfig(
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
    level=logging.DEBUG, handlers=[filehandler])

logger = logging.getLogger('spam_application')

for i in range(100):
    logger.debug("a log line, number: %s", i)

And proof that it works

❯❯❯ python3.4 main.py
❯❯❯ cat app.log*
2014-06-11 11:50:44,871 - spam_application - DEBUG - a log line, number: 91
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 92
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 93
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 94
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 95
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 96
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 97
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 98
2014-06-11 11:50:44,872 - spam_application - DEBUG - a log line, number: 99

Upvotes: 5

Related Questions