CODEWITHSUNDEEP
pythonmatplotlibplotuncertainty
patapouf_ai
patapouf_ai

Reputation: 18723

Plotting shaded uncertainty region in line plot in matplotlib when data has NaNs

I would like a plot which looks like this: plot with uncertainty

I am trying to do this with matplotlib:

fig, ax = plt.subplots()

with sns.axes_style("darkgrid"):
    for i in range(5):
        ax.plot(means.ix[i][list(range(3,104))], label=means.ix[i]["label"])
        ax.fill_between(means.ix[i][list(range(3,104))]-stds.ix[i][list(range(3,104))], means.ix[i][list(range(3,104))]+stds.ix[i][list(range(3,104))])
    ax.legend()

I want the shaded region to be the same colour as the line in the centre. But right now, my problem is that means has some NaNs and fill_between does not accept that. I get the error

TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Any ideas on how I could achieve what I want? The solution doesn't need to use matplotlib as long as it can plot my series of points with their uncertainties for multiple series.

Upvotes: 28

Views: 46579

Answers (2)

patapouf_ai
patapouf_ai

Reputation: 18723

Ok. So one of the problem was that the dtype of my data was object and not float and this caused fill_between to fail when it looked to see if the numbers were finite. I finally managed to do it by (a) converting to float and then (b) to solve the problem of the matching colours for uncertainty and line, to use a colour palette. So I have:

import seaborn as sns
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
fig, ax = plt.subplots()
clrs = sns.color_palette("husl", 5)
with sns.axes_style("darkgrid"):
    epochs = list(range(101))
    for i in range(5):
        meanst = np.array(means.ix[i].values[3:-1], dtype=np.float64)
        sdt = np.array(stds.ix[i].values[3:-1], dtype=np.float64)
        ax.plot(epochs, meanst, label=means.ix[i]["label"], c=clrs[i])
        ax.fill_between(epochs, meanst-sdt, meanst+sdt ,alpha=0.3, facecolor=clrs[i])
    ax.legend()
    ax.set_yscale('log')

which gave me the following result: enter image description here

Upvotes: 37

Diziet Asahi
Diziet Asahi

Reputation: 40737

You could simply drop the NaNs from your means DataFrame and plot that resulting dataframe instead?

In the example below, I tried to get close to your structure, I have a means DataFrame with some NaN sprinkled around. I suppose the stds DataFrame probably has NaN at the same locations, but in this case it doesn't really matter, I drop the NaN from means to get temp_means and I use the indices left in temp_means to extract the std values from stds.

The plots show the results before (top) and after (bottom) dropping the NaNs

x = np.linspace(0, 30, 100)
y = np.sin(x/6*np.pi)
error = 0.2

means = pd.DataFrame(np.array([x,y]).T,columns=['time','mean'])
stds = pd.DataFrame(np.zeros(y.shape)+error)

#sprinkle some NaN in the mean
sprinkles = means.sample(10).index
means.loc[sprinkles] = np.NaN


fig, axs = plt.subplots(2,1)

axs[0].plot(means.ix[:,0], means.ix[:,1])
axs[0].fill_between(means.ix[:,0], means.ix[:,1]-stds.ix[:,0], means.ix[:,1]+stds.ix[:,0])

temp_means = means.dropna()

axs[1].plot(temp_means.ix[:,0], temp_means.ix[:,1])
axs[1].fill_between(temp_means.ix[:,0], temp_means.ix[:,1]-stds.loc[temp_means.index,0], temp_means.ix[:,1]+stds.loc[temp_means.index,0])


plt.show()

enter image description here

Upvotes: 3

Related Questions