How to set logging level in twisted?

Question

I am using autobahn package with twisted which shows debug message for me every connect to websocket. I tried to switch logging level to info but had no success:

import logging
logging.basicConfig(level=logging.INFO)

Is there an easy way to switch log level?

Updated.

Here is the twisted_service.py:

from twisted.application import service
from twisted.logger import Logger
import logging
logging.basicConfig(level=logging.INFO)


class WebsocketService(service.Service):
    log = Logger()

    def startService(self):
        service.Service.startService(self)
        self.log.debug('start service')

application = service.Application("ws")

ws_service = WebsocketService()
ws_service.setServiceParent(application)

I run it using twistd script: twistd -noy twisted_service.py And i get a message:

2018-03-03T10:45:22+0500 [builtin.WebsocketService#debug] start service

logging.basicConfig didn't help.

Answer 1

In this example snippet, I demonstrate a scenario of integrating Twisted logging into another (custom) logging system (the code for which I've omitted). In this case, the other logger is managing a file object, stdout/err redirects, file rotations etc. I just wanted to to have Twisted supplement that other system and send it's messages over that effectively.

I additionally show here how to using log level filtering in Twisted, and how to correctly start and stop their logger as needed.

As shown in some other answers, you need to create a new "Observer" bound to a log file object, with a "Filter" to restrict the logging level. If you want to start/stop/restart - then note you once Twisted logging has previously been "started", one needs to redirect that output to a new file (via a new observer), rather than "start" their logging again. That prevents warnings which otherwise appear in the log if you attempt to "start" Twisted logging more than once.

I also point out that a call to theLogPublisher.removeObserver, is harmless to call if that observer was already removed (or never added). So, holding on to an observer reference after stopping, is in fact desired here for a simple logic flow, despite perhaps wanting to set such to None - even if that might equate to a tiny "waste" of memory in certain contexts.

from twisted.python import log as twistedlog
from twisted.logger import( FilteringLogObserver, textFileLogObserver,
                            LogLevelFilterPredicate, LogLevel )

twistedLogObserver = None

def startTwistedLogging( logFile, level, isStdMsgHdlr=False ):
    global twistedLogObserver
    observer = FilteringLogObserver( textFileLogObserver( logFile ),
                    predicates=[ LogLevelFilterPredicate( level ) ] )
    if twistedLogObserver:
        twistedlog.theLogPublisher.addObserver( observer )
    else: 
        twistedlog.startLoggingWithObserver( observer, setStdout=isStdMsgHdlr )
    twistedLogObserver = observer

def stopTwistedLogging():
    if twistedLogObserver:
        twistedlog.theLogPublisher.removeObserver( twistedLogObserver )

Answer 2

I used kalombo's answer to figure out a solution for my own project, which I'll share the meat of here since I couldn't find much accessible info on this subject, and I still had to figure out a few things. This example allows us to log to both stdout and a file, and choose a different log level for each.

(Thanks also to https://stackoverflow.com/a/46651223/213445 regarding logging to multiple files.)

Here's the breakdown:

• The variable "logtargets" is a list of FilteringLogObserver instances.
• Each FilteringLogObserver wraps a textFileLogObserver, and imposes a LogLevelFilterPredicate. We do this once for stdout and once for our "logfile".
• The LogLevelFilterPredicate conveys a maximum LogLevel to FilteringLogObserver through its "predicates" argument.
• The "predicates" argument to FilteringLogObserver must be iterable, so we wrap LogLevelFilterPredicate in a list when passing it.
• At the end, we pass our "logtargets" list to globalLogBeginner.beginLoggingTo().
• All future Twisted Logger instances will point to both of our log targets.
• We can have multiple instances of Logger, one for each subsystem of our program, and for each one we give it a single argument, which is the namespace for log lines from that subsystem.

import sys
from twisted.logger import Logger, LogLevel, LogLevelFilterPredicate, \
    textFileLogObserver, FilteringLogObserver, globalLogBeginner

logfile = open("application.log", 'a')
logtargets = []

# Set up the log observer for stdout.
logtargets.append(
    FilteringLogObserver(
        textFileLogObserver(sys.stdout),
        predicates=[LogLevelFilterPredicate(LogLevel.warn)]
    )
)

# Set up the log observer for our log file. "debug" is the highest possible level.
logtargets.append(
    FilteringLogObserver(
        textFileLogObserver(logfile),
        predicates=[LogLevelFilterPredicate(LogLevel.debug)]
    )
)

# Direct the Twisted Logger to log to both of our observers.
globalLogBeginner.beginLoggingTo(logtargets)

# Start a logger with a namespace for a particular subsystem of our application.
log = Logger("mynamespace")

log.warn("Goes to both outputs.") # Outputs will start with [mynamespace#warn]
log.error("Also go to both outputs; error is higher priority than warn.")
log.debug("Only goes to log file.")
log.info("Also only goes to log file; info is lower priority than warn.")

Answer 3

I found the solution:

import sys
from twisted.application import service
from twisted.logger import LogLevelFilterPredicate, LogLevel
from twisted.logger import textFileLogObserver, FilteringLogObserver


class WebsocketService(service.Service):
    log = Logger()

    def startService(self):
        service.Service.startService(self)
        self.log.debug('start service')

application = service.Application("ws")

ws_service = WebsocketService()
ws_service.setServiceParent(application)

info_predicate = LogLevelFilterPredicate(LogLevel.info)
log_observer = FilteringLogObserver(textFileLogObserver(sys.stdout), predicates=info_predicate)

application.setComponent(ILogObserver, log_observer)

Answer 4

To find all loggers and their corresponding names execute:

logging.Logger.manager.loggerDict.keys()

Try to set it explicitly by getting the noisy logger instance:

import logging

noisyLogger = logging.getLogger("autobahn")  # adjust logger name
# noisyLogger.getEffectiveLevel()  # indicates the current effective level
noisyLogger.setLevel(logging.INFO)