CODEWITHSUNDEEP
pythonlogging
Kingname
Kingname

Reputation: 1362

Why Python's RotatingFileHandler of logging module will rename log file instead of create a new one?

I have read the handlers.pyof Python's logging module, and found that the logic of rotating log is unreasonable. The follow picture is its comments: enter image description here

I think this logic is unreasonable especially in Windows: If some other processes hold the handler of the log file, then the renaming will fail. For example, I have two processes, one process find it is time to rotate file, so it close its own handler and rename the log. But this time, the log file's handler is held by another process, so an IO Exception will be raised because the log file can not be renamed now. That is why we say logging module is unsafe in multi-process.

My question is: Why not create another log file directly? Why does it need to rename the log file?

Suppose the log named console.log, I think the logic should be like this:

if there is not console.log in destination folder:
    create console.log and logging info
else:
    add the suffix to the log file name.(e.g.: console.log.1)
    create the file and logging info

if the log file gets filled up:
    close the handler of the log file
    increase the suffix number
    create console.log.`newsuffix`(e.g.:console.log.2) and continue to logging info

In this logic, there is no need to rename the file, so there will not raise the IO Exception.

Can you find some bugs if the rotatingFileHandler of Python's logging module use my logic?

Upvotes: 1

Views: 919

Answers (2)

Cesar
Cesar

Reputation: 717

The main problem with your proposal is that the actual logging file always will be changing (first will be console.log, later will be console.log.1, after that console.log.2 etc...) so, all the programs that will read the logs, they will have to implement logic to find what is the newer file, the main problem is that the software reading logs will need to constantly scan the logs folder to find if a new file is created and start to read logs from it.

Upvotes: 1

pferate
pferate

Reputation: 2107

This is so that console.log will always be the latest log file, you don't need any extra logic to find the newest log file. This has been standard practice for ages.

I know that, at least on Linux/Unix, when a process opens a file, it's filename gets translated to a inode value, and that is used as a reference, not it's filename.

Any processes that had the file open previously will still be pointed to that inode, until their own lock is refreshed or closed. With this in mind, moving or deleting a file does nothing to any processes that previously had the file handle open, but everybody else will not see the old file (at least by filename).

Upvotes: 1

Related Questions