CODEWITHSUNDEEP
pythonmultithreadinglogging
russell
russell

Reputation: 708

can children of python loggers be deleted?

I'm writing a framework in python to implement some servers. They run continuously, waiting for jobs, and when one or more is found they can either process it, start one or more threads to process it, or start a subprocess to handle it. When it is looking for a job log data should go into one file, which will rotate daily. Log data for each job will go into a separate job log file with name based on the job ID.

This is the first time I've really tried using python logging. What is the "right" way to do this? One thought would be to add a child logger for each job, but there does not appear to be a way to remove child loggers (I think it is not hard to do by mucking with the library code, but I assume there is no remove_child() method for a reason).

The other way I'm thinking of is to set up a handler for each job, and give each one a filter based on the current thread. This way all handlers would be filtered out except for the one in the active thread, which would log to the job log file. This way would be easy to remove handlers when they finish. This would probably work - I expect there won't be more than around 10 threads active at once for any job, so running a filter for each isn't that hard. Is this the best way? Or is it something else I haven't thought of?

Upvotes: 1

Views: 183

Answers (1)

Vinay Sajip
Vinay Sajip

Reputation: 99485

Loggers relate to areas in your application/places in your code, so it doesn't really make sense to add and remove loggers. The approach you described with handlers for each job and filters to ensure that messages are only logged to one file would work, and any code you called in third-party libraries would also have its messages directed to the appropriate place. While not exactly the same case, I described how this would work for another multi-threaded scenario - web applications - in this post.

Upvotes: 1

Related Questions