Alex Poca
Alex Poca

Reputation: 2566

Pandas groupby: group by semester

I need to group data by semesters but there is no frequency tag available here

2QS (2 quarters from start) and 6MS (6 months from start) won't do because they will start in different moments, according to the first datetime in my dataframe. (Quite counterintuitive and prone to errors, IMHO: I didn't see this issue till I used a different dataset that began in May instead of January...)

from datetime import *
import pandas as pd
import numpy as np

df = pd.DataFrame()

days = pd.date_range(start="2017-05-17", 
                     end="2017-11-29",
                    freq="1D")
df = pd.DataFrame({'DTIME': days, 'DATA': np.random.randint(50, high=80, size=len(days))})
df.set_index('DTIME', inplace=True)

grouped = df.groupby(pd.Grouper(freq='2QS'))
print("Groups date start:")
for dtime, group in grouped:
    print dtime
    # print(group)

returns

Groups date start:
2017-04-01 00:00:00   <== because my first datetime is in May, 2017
2017-10-01 00:00:00

instead of:

Groups date start:
2017-01-01 00:00:00   <== I want the semesters referred to the year!
2017-06-01 00:00:00

As a possible workaround I created two new columns in my dataframe and then group according to them:

      df["year"] = df.index.year.astype(int)
      df["semester"] = df.index.month.astype(int)
      df["semester"] = df["semester"] - 1
      df["semester"] = df["semester"] // 6
      grouped = df.groupby(["year", "semester"])

Is this the only way to to this?

There are two other little questions, just for the sake of curiosity and not worth an indipendent stackoverflow question:

  1. why a tag W (end of week) is available, but WS (start of week) is not?

  2. how to write this in a single line?

      df["semester"] = df.index.month.astype(int)
      df["semester"] = df["semester"] - 1
      df["semester"] = df["semester"] // 6
    

Upvotes: 3

Views: 1938

Answers (1)

jezrael
jezrael

Reputation: 862921

The closest are anchored-offsets, but for month it missing.

And for second:

df["semester"] =  (df.index.month.astype(int) - 1) // 6

Or without creating new column:

years = df.index.year.astype(int)
semes = (df.index.month.astype(int) - 1) // 6
grouped = df.groupby([years, semes])

Upvotes: 3

Related Questions