CODEWITHSUNDEEP
pythonpandasseabornheatmapdatetimeindex
matthme
matthme

Reputation: 263

Heatmap with pandas DateTimeIndex on both axis

I would like to make a heatmap from a pandas DataFrame (or Series) with DateTimeIndex so that I have hours on the x-axis and days on the y-axis, both ticklabels displayed in DateTimeIndex style.

If I do the following:

    import pandas as pd
    import numpy as np
    import seaborn as sns

    df = pd.DataFrame(np.random.randint(10, size=4*24*200))
    df.index = pd.date_range(start='2019-02-01 11:30:00', periods=200*24*4, freq='15min')

    df['minute'] = df.index.hour*60 + df.index.minute
    df['dayofyear'] = df.index.month + df.index.dayofyear

    df = df.pivot(index='dayofyear', columns='minute', values=df.columns[0])
    sns.heatmap(df)

The index obviously loses the DateTime format:

enter image description here

What I instead want is something like this (which I achieved with a complicated, not generalizable function that apparently doesn't even work properly):

enter image description here

Does someone know a neat way to create this kind of heatmap with python?

EDIT:

The function I created:

    def plot_heatmap(df_in, plot_column=0, figsize=(20,12), vmin=None, vmax=None, cmap='jet', xlabel='hour (UTC)', ylabel='day', rotation=0, freq='5s'):
        '''
        Plots heatmap with date labels

        df_in:    pandas DataFrame od pandas Series
        plot_column:  column to plot if DataFrame has multiple columns

        ...

        '''

        # convert to DataFrame in case a Series is passed:
        try:
            df_in = df_in.to_frame()
        except AttributeError:
            pass
        
        # make copy in order not to overrite input (in case input is an object attribute)
        df = df_in.copy()

        # pad missing dates:
        idx = pd.date_range(df_in.index[0], df_in.index[-1], freq=freq)
        df = df.reindex(idx, fill_value=np.nan)


        df['hour'] = df.index.hour*3600 + df.index.minute*60 + df.index.second
        df['dayofyear'] = df.index.month + df.index.dayofyear

        # Create mesh for heatmap plotting:
        pivot = df.pivot(index='dayofyear', columns='hour', values=df.columns[plot_column])

        # plot
        plt.figure(figsize=figsize)
        sns.heatmap(pivot, cmap=cmap)

        # set xticks
        plt.xticks(np.linspace(0,pivot.shape[1],25), labels=range(25))
        plt.xlabel(xlabel)

        # set yticks
        ylabels = []
        ypositions = []

        day0 = df['dayofyear'].unique().min()
        for day in df['dayofyear'].unique():
            day_delta = day-day0
            # create pandas Timestamp
            temp_tick = df.index[0] + pd.Timedelta('%sD' %day_delta)
            # check wheter tick shall be shown or not
            if temp_tick.day==1 or temp_tick.day==15:
                temp_tick_nice = '%s-%s-%s' %(temp_tick.year, temp_tick.month, temp_tick.day)
                ylabels.append(temp_tick_nice)
                ypositions.append(day_delta)


        plt.yticks(ticks=ypositions, labels=ylabels, rotation=0)
        plt.ylabel(ylabel)

Upvotes: 1

Views: 949

Answers (2)

matthme
matthme

Reputation: 263

The best solution I found now that also works if the frequency of the DatetimeIndex is <1min is the following:

import pandas as pd
import numpy as np
import seaborn as sns

freq = '30s'

df = pd.DataFrame(np.random.randint(10, size=4*24*200*20))
df.index = pd.date_range(start='2019-02-01 11:30:00', periods=200*24*4*20, freq=freq)

df['hour'] = df.index.strftime('%H:%M:%S')
df['dayofyear'] = df.index.date


df = df.pivot(index='dayofyear', columns='hour', values=df.columns[0])
df.columns = pd.DatetimeIndex(df.columns).strftime('%H:%M')
df.index = pd.DatetimeIndex(df.index).strftime('%m/%Y')

xticks_spacing = int(pd.Timedelta('2h')/pd.Timedelta(freq))
ax = sns.heatmap(df, xticklabels=xticks_spacing, yticklabels=30)
plt.yticks(rotation=0)

Which produces this result:

enter image description here

The only flaw yet is that the month ticks positions are not well defined and precise with this method...

Upvotes: 0

Quang Hoang
Quang Hoang

Reputation: 150745

The date format going away because you did:

df['dayofyear'] = df.index.month + df.index.dayofyear

Here, both series are integers, so df['dayofyear'] is also integer-typed.

Instead, do:

df['dayofyear'] = df.index.date

Then you get as output:

enter image description here

Upvotes: 1

Related Questions