CODEWITHSUNDEEP
pythonloggingjupyter-notebook
Deskjokey
Deskjokey

Reputation: 610

How to toggle Jupyter Notebook display logging

I am trying to apply logging to Jupyter Notebook's display(df) functionality. How do I edit the last line of my code so that the dataframe is only displayed if logging INFO is active? Currently it's shown whether the logging is active or inactive.

import pandas as pd

logging.basicConfig(level=logging.INFO, format=' %(asctime)s - %(levelname)s - %(message)s')

filename = 'test.csv'
df=pd.read_csv(filename, index_col=None, encoding='utf-8')

logging.info(display(df))

Upvotes: 2

Views: 9805

Answers (2)

Simon Rochester
Simon Rochester

Reputation: 136

The currently accepted answer shows the str representation of the dataframe, whereas I believe the OP was interested in logging the IPython html repr. This can be achieved with a custom Handler:

import logging

class DisplayHandler(logging.Handler):
    def __init__(self):
        super().__init__()
        self.addFilter(lambda record: hasattr(record.msg, '_repr_html_'))
    
    def emit(self, record):
        display(record.msg)

logging.basicConfig()
logger = logging.getLogger()
logger.addHandler(DisplayHandler())

Now display depends on log level:

import pandas as pd
df = pd.DataFrame({'a':[1,2,3], 'b':[4,5,6], 'c':[7,8,9]})

logger.info(df)

shows nothing, but for a warning, the dataframe is displayed:

logger.warning(df)

enter image description here

Now both the default and the custom handlers are handling the dataframe; to filter it from the default handler we can do

logger.handlers[0].addFilter(lambda record: not hasattr(record.msg, '_repr_html_'))

Now we have

logger.warning(df)

enter image description here

The same handler works for any object that has a html repr, for example, matplotlib figures:

import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure(figsize=(3, 2))
x = np.arange(0., 10., 0.2)
plt.plot(x, np.sin(x))

logger.warning(fig)

enter image description here

Upvotes: 0

keineahnung2345
keineahnung2345

Reputation: 2701

Currently it's shown whether the logging is active or inactive.

I guess you are saying that when you change the argument level of logging.basicConfig, the result doesn't change along with it. That happens to me, too. After checking logging - Doc and Logging basicConfig not creating log file when i run in pycharm?, I am able to come out a solution:

for handler in logging.root.handlers[:]:
    logging.root.removeHandler(handler)

logging.basicConfig(format=' %(asctime)s - %(levelname)s - %(message)s')
df = pd.DataFrame({'a':[1,2,3], 'b':[4,5,6], 'c':[7,8,9]})

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logger.debug("\n"+str(df))
logger.info("\n"+str(df))
logger.warning("\n"+str(df))
logger.error("\n"+str(df))
logger.critical("\n"+str(df))

You can play with logger.setLevel(logging.DEBUG) to see if it works.

And since display(df) would be called anyway just like @AChampion says, I use "\n"+str(df) to replace display(df).

Output:

 2019-01-18 14:20:47,710 - DEBUG - 
   a  b  c
0  1  4  7
1  2  5  8
2  3  6  9
 2019-01-18 14:20:47,715 - INFO - 
   a  b  c
0  1  4  7
1  2  5  8
2  3  6  9
 2019-01-18 14:20:47,720 - WARNING - 
   a  b  c
0  1  4  7
1  2  5  8
2  3  6  9
 2019-01-18 14:20:47,724 - ERROR - 
   a  b  c
0  1  4  7
1  2  5  8
2  3  6  9
 2019-01-18 14:20:47,728 - CRITICAL - 
   a  b  c
0  1  4  7
1  2  5  8
2  3  6  9

Upvotes: 6

Related Questions