How to dynamically update a plot in a loop in IPython notebook (within one cell)

Question

Environment: Python 2.7, Matplotlib 1.3, IPython notebook 1.1, Linux, and Chrome. The code is in one single input cell, using --pylab=inline.

I want to use IPython notebook and Pandas to consume a stream and dynamically update a plot every five seconds.

When I just use a print statement to print the data in text format, it works perfectly fine: the output cell just keeps printing data and adding new rows. But when I try to plot the data (and then update it in a loop), the plot never shows up in the output cell. But if I remove the loop, and just plot it once, it works fine.

Then I did some simple test:

i = pd.date_range('2013-1-1',periods=100,freq='s')
while True:
    plot(pd.Series(data=np.random.randn(100), index=i))
    #pd.Series(data=np.random.randn(100), index=i).plot() also tried this one
    time.sleep(5)

The output will not show anything until I manually interrupt the process (Ctrl + M + I). And after I interrupt it, the plot shows correctly as multiple overlapped lines. But what I really want is a plot that shows up and gets updated every five seconds (or whenever the plot() function gets called, just like what print statement outputs I mentioned above, which works well). Only showing the final chart after the cell is completely done is not what I want.

I even tried to explicitly add the draw() function after each plot(), etc. None of them works. How can I dynamically update a plot by a for/while loop within one cell in IPython notebook?

Answer 1

Minimal modern solution for notebooks in Jupyter Lab (version 3.6.1):

from matplotlib import pyplot as plt

fig, ax = plt.subplots(1)
ax.set(xlabel=f'Epochs', ylabel='Value',
       title='Dynamics')

for i in range(n_step):
   ...
   ax.plot(...)
   
   if i==0:
      ax.legend(labels, loc='upper right')
   else:
      display(fig, clear=True);

Notice! No need for neither %matplotlib inline nor from IPython.display import clear_output, display because by default a new notebook already has a function with signature:

display(
    *objs,
    include=None,
    exclude=None,
    metadata=None,
    transient=None,
    display_id=None,
    raw=False,
    clear=False,
    **kwargs,
)

Note the keyword argument clear! Setting it to True makes a deal.

Answer 2

A nice solution has been proposed by @BlackHC in a related post. It consists in using IPython.display.display with display_id=True to obtain a handle and use the update() method on it.

For instance,

import time

from IPython.display import display
from matplotlib import pyplot as plt

import numpy as np


hdisplay_img = display(display_id=True)
hdisplay_txt = display(display_id=True)

fig = plt.figure()
ax = fig.add_subplot(111)
im = ax.imshow(np.random.random((10,10,3)))
plt.close()

def update(i):
    im.set_data(np.random.random((10,10,3)))
    ax.add_image(im)
    hdisplay_img.update(fig)
    hdisplay_txt.update(f"update {i}")

for f in range(10):
    update(f)
    time.sleep(1)

Answer 3

You can do it like this. It accepts x,y as list and output a scatter plot plus a linear trend on the same plot.

from IPython.display import clear_output
from matplotlib import pyplot as plt
%matplotlib inline
    
def live_plot(x, y, figsize=(7,5), title=''):
    clear_output(wait=True)
    plt.figure(figsize=figsize)
    plt.xlim(0, training_steps)
    plt.ylim(0, 100)
    x = [float(i) for i in x]
    y = [float(i) for i in y]
    
    if len(x) > 1:
        plt.scatter(x,y, label='axis y', color='k') 
        m, b = np.polyfit(x, y, 1)
        plt.plot(x, [x * m for x in x] + b)

    plt.title(title)
    plt.grid(True)
    plt.xlabel('axis x')
    plt.ylabel('axis y')
    plt.show();

You just need to call live_plot(x, y) inside a loop. Here's how it looks:

Answer 4

Adding a label to the other solutions posted here will keep adding new labels in every loop. To deal with that, clear the plot using clf.

For example:

for t in range(100):
   if t % refresh_rate == 0:

     plt.clf()
     plt.plot(history['val_loss'], 'r-', lw=2, label='val')
     plt.plot(history['training_loss'], 'b-', lw=1, label='training')
     plt.legend()
     display.clear_output(wait=True)
     display.display(plt.gcf())

Answer 5

Use the IPython.display module:

%matplotlib inline
import time
import pylab as pl
from IPython import display
for i in range(10):
    pl.plot(pl.randn(100))
    display.clear_output(wait=True)
    display.display(pl.gcf())
    time.sleep(1.0)

Answer 6

I tried many methods, but I found this as the simplest and the easiest way -> to add clear_output(wait=True), for example,

from IPython.display import clear_output

for i in range(n_iterations):
     clear_output(wait=True)
     x = some value
     y = some value
     plt.plot(x, y, '-r')
     plt.show()

This overwrites on the same plot, and gives an illusion of plot animation

Answer 7

A couple of improvement's on HYRY's answer:

• call display before clear_output so that you end up with one plot, rather than two, when the cell is interrupted.
• catch the KeyboardInterrupt, so that the cell output isn't littered with the traceback.

import matplotlib.pylab as plt
import pandas as pd
import numpy as np
import time
from IPython import display
%matplotlib inline

i = pd.date_range('2013-1-1',periods=100,freq='s')

while True:
    try:
        plt.plot(pd.Series(data=np.random.randn(100), index=i))
        display.display(plt.gcf())
        display.clear_output(wait=True)
        time.sleep(1)
    except KeyboardInterrupt:
        break

Answer 8

You can further improve this by adding wait=True to clear_output:

display.clear_output(wait=True)
display.display(pl.gcf())

Answer 9

Try to add show() or gcf().show() after the plot() function. These will force the current figure to update (gcf() returns a reference for the current figure).