CODEWITHSUNDEEP
pythonlogging
Jace Browning
Jace Browning

Reputation: 12692

How can I include the relative path to a module in a Python logging statement?

My project has a subpackage nested under the root package like so:

My goal is to get logging records formatted like:

mypackage/topmodule.py:123: First log message
mypackage/subpackage/nested.py:456: Second log message

so that the paths become clickable in my terminal.

I've tried the following formats.

Is there a logging pattern I can pass to logging.basicConfig(format='???') that wil produce the logging records I desire?

Upvotes: 10

Views: 3319

Answers (3)

Marine Galantin
Marine Galantin

Reputation: 2289

Filters can be used but I think a more pythonic way of doing this would be to extend a Formatter:

class CustomFormatter(logging.Formatter):
    def __init__(self, root_path, fmt, datefmt=None):
        super().__init__(fmt, datefmt)
        self.root_path = root_path

    def format(self, record):
        # Replace the full pathname with the relative path
        record.pathname = os.path.relpath(record.pathname, self.root_path)
        return super().format(record)

Upvotes: 2

elton fernando
elton fernando

Reputation: 151

This produces the same result.

import os
import logging

    class PackagePathFilter(logging.Filter):
        def filter(self, record):
            record.pathname = record.pathname.replace(os.getcwd(),"")
            return True

add handler

handler.addFilter(PackagePathFilter())

and together with '%(pathname)s:%(lineno)s: %(message)s' as the format your log.

Upvotes: 3

Martijn Pieters
Martijn Pieters

Reputation: 1125368

You'd have to do additional processing to get the path that you want here.

You can do such processing and add additional information to log records, including the 'local' path for your own package, by creating a custom filter.

Filters don't actually have to do filtering, but they do get access to all log records, so they are a great way of updating records with missing information. Just make sure you return True when done:

import logging
import os
import sys


class PackagePathFilter(logging.Filter):
    def filter(self, record):
        pathname = record.pathname
        record.relativepath = None
        abs_sys_paths = map(os.path.abspath, sys.path)
        for path in sorted(abs_sys_paths, key=len, reverse=True):  # longer paths first
            if not path.endswith(os.sep):
                path += os.sep
            if pathname.startswith(path):
                record.relativepath = os.path.relpath(pathname, path)
                break
        return True

This finds the sys.path entry that's the parent dir for the pathname on the logrecord and adds a new relativepath entry on the log record. You can then use %(relativepath)s to include it in your log.

Add the filter to any handler that you have configured with your custom formatter:

handler.addFilter(PackagePathFilter())

and together with '%(relativepath)s:%(lineno)s: %(message)s' as the format your log messages will come out like:

mypackage/topmodule.py:123: First log message
mypackage/subpackage/nested.py:456: Second log message

(actual output, except I altered the line numbers on that).

Upvotes: 9

Related Questions