How to plot confidence interval of a time series data in Python?

Question

There are multiple questions exist on this area, however I can not use them to solve my question. I have a data sample and I want to create the confidence interval for its curve. Here, I provide a simple example:

import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt

mean, lower, upper = [],[],[]
ci = 0.2
for i in range (20):

    a = np.random.rand(100) # this is the output

    MEAN = np.mean(a)
    mean.append(MEAN)
    std = np.std(a)
    Upper = MEAN+ci*std
    Lower = MEAN-ci*std

    lower.append(Lower)
    upper.append(Upper)


 plt.figure(figsize=(20,8))
 plt.plot(mean,'-b', label='mean')
 plt.plot(upper,'-r', label='upper')
 plt.plot(lower,'-g', label='lower')

 plt.xlabel("Value",   fontsize = 30)
 plt.ylabel("Loss", fontsize = 30)
 plt.xticks(fontsize= 30) 
 plt.yticks(fontsize= 30) 
 plt.legend(loc=4, prop={'size': 30})

In the above example, I drew %80 confidence interval. I have two questions:

1- Could you please tell me that this way of calculating and plotting the confidence interval is true?

2- I want to color the shadow area of the confidence interval. I have attached a figure, I want some thing like that. Could you please tell me if you have any solution? Thanks for your help.

Answer 1

I'm not qualified to answer question 1, however the answers to this SO question produce different results from your code.

As for question 2, you can use matplotlib fill_between to fill the area between two curves (the upper and lower of your example).

import numpy as np
import matplotlib.pyplot as plt
import scipy.stats

# https://stackoverflow.com/questions/15033511/compute-a-confidence-interval-from-sample-data
def mean_confidence_interval(data, confidence=0.95):
    a = 1.0 * np.array(data)
    n = len(a)
    m, se = np.mean(a), scipy.stats.sem(a)
    h = se * scipy.stats.t.ppf((1 + confidence) / 2., n-1)
    return m, m-h, m+h

mean, lower, upper = [],[],[]
ci = 0.8
for i in range (20):
    a = np.random.rand(100) # this is the output
    m, ml, mu = mean_confidence_interval(a, ci)
    mean.append(m)
    lower.append(ml)
    upper.append(mu)

plt.figure()
plt.plot(mean,'-b', label='mean')
plt.plot(upper,'-r', label='upper')
plt.plot(lower,'-g', label='lower')
# fill the area with black color, opacity 0.15
plt.fill_between(list(range(len(mean))), upper, lower, color="k", alpha=0.15)

plt.xlabel("Value")
plt.ylabel("Loss")
plt.legend()