CODEWITHSUNDEEP
pythonpandasjoindataframemerge
ayyayyekokojambo
ayyayyekokojambo

Reputation: 1245

Pandas Merge Two Different sized Dataframe on Datetime

I have two dataframes that i want to merge on date column.

First dataframe holds datetimes:

    DateTime,Date,Hour

    2015-01-01 00:00:00 | 2015-01-01 | 00 
    2015-01-01 00:00:01 | 2015-01-01 | 01 

    ...

    2015-01-01 23:00:00 | 2015-01-01 | 23

Second one holds it daily basis:

> Date,Col3
> 
> 2015-01-01 | daily something1
> 
> 2015-01-02 | daily something2

--

I want to merge on Date column so that 24 hours in a date will have same daily features taken by second dataframe.

2015-01-01 00:00:00 | 2015-01-01 | 00 |  daily something1

2015-01-01 01:00:00 | 2015-01-01 | 01 |  daily something1

...

2015-01-02 00:00:00 | 2015-01-01 | 23|  daily something2

It can be done by writing some code, but can i do this with using join or merge? tried to do it with left,right join but couldnt done it.

Upvotes: 1

Views: 1542

Answers (1)

3novak
3novak

Reputation: 2544

Let's merge the following two dataframes in the manner you described. I don't know if there's a nice oneliner to accomplish this, and I'd like to see one, but this method works.

import pandas as pd

df = pd.DataFrame({'DATE': pd.date_range(start='2016-01-01 00:00:00',
                                         freq='12H', periods=10)})
df2 = pd.DataFrame({'DATE': pd.date_range(start='2016-01-01',
                                          freq='D', periods=5),
                    'VALUE': range(0,5)})

# extract the date from each column
df['DATE_DAY'] = df['DATE'].dt.date
# even though the df2 DATE column only shows the date, it's still in
# a different type (datetime64[ns]), so we have to convert it as well
df2['DATE_DAY'] = df2['DATE'].dt.date

tmp = df.merge(df2, on='DATE_DAY')
>>> tmp
               DATE_x     DATE_y    DATE_DAY  VALUE
0 2016-01-01 00:00:00 2016-01-01  2016-01-01      0
1 2016-01-01 12:00:00 2016-01-01  2016-01-01      0
2 2016-01-02 00:00:00 2016-01-02  2016-01-02      1
3 2016-01-02 12:00:00 2016-01-02  2016-01-02      1
4 2016-01-03 00:00:00 2016-01-03  2016-01-03      2
5 2016-01-03 12:00:00 2016-01-03  2016-01-03      2
6 2016-01-04 00:00:00 2016-01-04  2016-01-04      3
7 2016-01-04 12:00:00 2016-01-04  2016-01-04      3
8 2016-01-05 00:00:00 2016-01-05  2016-01-05      4
9 2016-01-05 12:00:00 2016-01-05  2016-01-05      4

Upvotes: 1

Related Questions