Custom HTTPExceptions with custom logging system in FastAPI Python

Question

I have problem with FastApi logging system. I created custom logs but when I try to raise HTTPException(status_code=[some code], details=[msg] the FastApi output is "INFO" level not "ERROR". So I decide to write custom HTTPException and here is my question, how can I do it witch my logging system?

log_file.py

import logging.handlers
import logging
import datetime
import yaml

class CustomFormatter(logging.Formatter):
    LOG_DIR = 'server_logs/'

    blue = '\x1b[38;5;39m'
    green = '\x1b[1;32m'
    yellow = '\x1b[38;5;226m'
    red = '\x1b[38;5;196m'
    bold_red = '\x1b[31;1m'
    reset = '\x1b[0m'

    data = '%(asctime)s | '
    level_name = '%(levelname)8s '
    file_and_message = '| (%(filename)s:%(lineno)d) | %(message)s'

    FORMATS = {
            logging.DEBUG: data + blue + level_name + reset + file_and_message,
            logging.INFO: data + green + level_name + reset + file_and_message,
            logging.WARNING: data + yellow + level_name + reset + file_and_message,
            logging.ERROR: data + red + level_name + reset + file_and_message,
            logging.CRITICAL: data + bold_red + level_name + reset + file_and_message
        }

    def format(self, record):
        log_fmt = self.FORMATS.get(record.levelno)
        formatter = logging.Formatter(log_fmt)
        return formatter.format(record)

class Logger:
    with open("server_config.yaml", "r+") as f:
        server_config = yaml.safe_load(f)
    logger = logging.getLogger()
    logger.setLevel(server_config['LOG_LEVEL_CONSOLE'].upper())
    logger.handlers = []

    # output custom log format in console
    console = logging.StreamHandler()
    console.setLevel(server_config['LOG_LEVEL'].upper())
    console.setFormatter(CustomFormatter())

    # save custom logs format to file
    today = datetime.date.today()
    save_in_file = logging.handlers.RotatingFileHandler(
        CustomFormatter.LOG_DIR +
        'Server_application{}.log'.format(today.strftime('%Y_%m_%d'))
    )
    save_in_file.setLevel(logging.DEBUG)
    save_in_file.setFormatter(CustomFormatter())

    # Add both handlers to the logger
    logger.addHandler(console)
    logger.addHandler(save_in_file)

this I get when I try to use HTTPException 2023-11-05 17:37:31,538 | INFO | (h11_impl.py:478) | 127.0.0.1:54249 - "GET /vehicle/all HTTP/1.1" 404 And like you see, this is output as a "INFO", I want to change it flexible, sometimes I would like to use "INFO" sometime "ERROR" etc.

I know that, I can use logger.error("message") but also I have couple files where I save this logs and every part of my application have his own file so Server will save his every output in his own file but I want to have correct response first. Second thing is that, HTTPException have "detail" what is returned to user and my logger.[logLevel] doesn't do it.

I try to write my own HTTPException using knowledge from internet but every time I tried I get something that isn't expected for me. for example I make something like this:

class CustomException(Exception):
    def __init__(self, name: str, info: str):
        self.status_code = name
        self.detail = info


@app.exception_handler(CustomException)
async def unicorn_exception_handler(request: Request, exc: CustomException):
    return HTTPException(
        status_code=404,
        detail={"message": f"some problem"},
    )


@app.get("/{name}")
async def read_unicorns(name: str):
    if name_true:
        logger.info("some info")
    raise UnicornException(name=name, info="some info")

Answer 1

You wrote you have a problem with FastApi logging system, I assume you mean the logging system you created and not the FastAPI logging system itself (uvicorn)?

Anyway, first of all I would recommend using the FastAPI logging system (uvicorn) and not writing your own, That way you keep the same format, and you can also add and change as needed

log_config.py

import logging
import uvicorn

# Define a custom log formatter class
class CustomLogFormatter(uvicorn.logging.DefaultFormatter):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

# Create custom record (for example request_id in my case)
    def format(self, record):
        record.__dict__["request_id"] = record.__dict__.get("request_id", "")
        # record.__dict__["levelprefix"] = record.__dict__.get("levelname", "").lower()
        return super().format(record)

FORMAT: str = "%(levelprefix)s %(asctime)s [%(threadName)s]  [%(name)s]  (%(request_id)s)  %(message)s"

def init_loggers(logger_name: str = "defualt-logger"):

    # create logger
    logger = logging.getLogger(f'{logger_name}')
    logger.setLevel(logging.DEBUG)

    # create console handler
    console_handler = logging.StreamHandler()
    console_handler.setLevel(logging.DEBUG)

    # create formatter (If you don't want a special record)
    # formatter = uvicorn.logging.DefaultFormatter(FORMAT, datefmt="%Y-%m-%d %H:%M:%S")
    
    # Create a CustomLogFormatter instance for your custom logs
    formatter_custom = CustomLogFormatter(FORMAT, datefmt="%Y-%m-%d %H:%M:%S")

    # add formatter to console_handler
    console_handler.setFormatter(formatter_custom)

    # add console_handler to logger
    logger.addHandler(console_handler)

    return logging.getLogger(logger_name)

call it in main.py

from app.log_config import init_loggers
log = init_loggers(logger_name="main-logger")

log.info(f"some data.")
log.warning(f"some data.")
log.error(f"some data.")

and it will look like this:

now for the INFO part, From what I understand, this log comes from the fastapi uvicorn default logs, like this one:

INFO:     172.18.0.4:46452 - "POST /auth/user-registration HTTP/1.1" 409 Conflict

Basically you can turn off uvicorn's default logger if you want, and you can implement it by yourself, add middleware, add_exception_handler..

I saw this medium article, there are some parts in there that might help you FastAPI Server Errors And Logs