CODEWITHSUNDEEP
pythongarbage-collection
capgelka
capgelka

Reputation: 99

How to prevent module from deleting by garbage collector or reimport it inside __del__ method

I wrote a wrapper class for logging module which has __del__method to delete log file at the end of the programm.

My programm copy this log files into archive from other class and it is important to have no log files in working folder before next programm execution. But I can't delete it from other module, because they are still hanlded with logger.

So I used __del__ method for this.

def __del__(self):
    if self.config_object.logfile is not None:
        import os
        try:
            os.unlink(self.config_object.logfile)
        except (WindowsError, AttributeError):
            pass

I reimport os because sometimes in has been destroed by garbage collector before my Logging class. And this work fine in python3.2

But after switching to python3.4 I saw errors like this:

  File "<frozen importlib._bootstrap>", line 2236, in _find_and_load
  File "<frozen importlib._bootstrap>", line 263, in __enter__
  File "<frozen importlib._bootstrap>", line 287, in _get_module_lock
  File "<frozen importlib._bootstrap>", line 173, in __init__
TypeError: 'NoneType' object is not callable

I tried to prevent os deleting instead of reimporting os but failed.

I tried this ideas: Creating cycle referencing with os inside. Adding field with os module inside my class. Calling gc.disabled() direclty. But none of them helped.

So I has two questions. First is how can one reimport os in python 3.4 with its new importlib while python programm is destroying by garbage collector. And second is how one can prevent the garbage collector from deleting os module.

P.S. I know that use context manager is better then __del__ and read questions about this iat SO. And of course I can change my program logic to avoid necessity of deleting log files. So this problem with log is just an example that awakened my curiosity. And if somebody can proof that both my goals is impossible it's ok and the best answer. But if an ugly and unpythonic solution exists I want to know about it even it is changing bytecode direclty someway or something like this.

Upvotes: 1

Views: 409

Answers (2)

Martijn Pieters
Martijn Pieters

Reputation: 1121664

You can bind a reference to the os.unlink() function as default argument, so you still have access to just the function when the hook runs:

def __del__(self, _os_unlink=os.unlink):
    if self.config_object.logfile is not None:
        import os
        try:
            _os_unlink(self.config_object.logfile)
        except (WindowsError, AttributeError):
            pass

However, take into account that there is never a guarantee that the hook is executed on Python shutdown.

Also, if you need to call other functions that happen to be implemented in pure python then you'll need to take into account that those functions too could be using dependencies that have been set to None at this time. You may have to either re-implement those functions with 'captured' references, or you'd have to restore the dependencies just for your call. For example, os.path.exists() requires os.stat() to still be available. Either re-implement the genericpath.exists() function with os.stat captured, or at least ensure that os.stat exists before calling the captured os.path.exists() function.

Upvotes: 2

Dunes
Dunes

Reputation: 40703

In your case it would be better to simply use a temporary file created using the tempfile module rather than trying to manage your own temporary file.

In the more abstract sense you make use of the fact that python guarantees a module's private globals (with a leading underscore) are deleted before its public globals. eg.

from os import unlink

class TempFileManager:
    def __init__(self):
        self.temp_files = []
    def __del__(self):
        for file_ in self.temp_files:
            # unlink still available as _temp_file_manager is deleted first
            unlink(file_)

_temp_file_manager = TempFileManager()

You may also wish to take a look at the atexit module rather than relying on __del__ semantics. eg.

import os
import atexit

file_ = ...

atexit.register(os.unlink, file_)

Upvotes: 0

Related Questions