CODEWITHSUNDEEP
pythonpandasdataframemerge
Denver Dang
Denver Dang

Reputation: 2615

Merge multiple data frames by date (remove duplicates)

I am trying to get a combined data frame from several data frames. Basically each single data frame is data for a month. So each has a date and some columns with some other values. However, the length of each data frame is not always the same, and some of the dates are repeated in some of them.

So what I would like to do is combine all my data frames (resulting in data for a year from 12 different data frames of each one month), so I end up with one data frame with no repeated dates. It should be mentioned that the columns are the same in each data frame. Just different values.

Can this be done in a simple way, or...?

Upvotes: 0

Views: 352

Answers (1)

Ruthger Righart
Ruthger Righart

Reputation: 4921

Import module.

import pandas as pd

Data example.

df1 = pd.DataFrame({'Date':['2020-6-19', '2020-6-20', '2020-7-20'], 'A':[1,2,3]})
df2 = pd.DataFrame({'Date':['2020-7-20', '2020-7-22'], 'A':[1,2]})
df1['Date'] = pd.to_datetime(df1['Date'])
df2['Date'] = pd.to_datetime(df2['Date'])

Concatenate your different DataFrames and drop duplicates on Date.

pd.concat([df1,df2]).drop_duplicates('Date').reset_index(drop=True)

Upvotes: 1

Related Questions