CODEWITHSUNDEEP
pythonpandas
user2775128
user2775128

Reputation: 407

pandas update dataframe by another dataframe with group by columns

I have two dataframe like this 

df1 = pd.DataFrame({'A': ['1', '2', '3', '4','5'],
                  'B': ['1', '1', '1', '1','1'],
                  'C': ['A', 'A1', 'A2', 'A3','A4'],
                  'D': ['B0', 'B1', 'B2', 'B3','B4'],
                  'E': ['A', 'A', 'S', 'S','S']})
df2 = pd.DataFrame({'A': ['1', '6', '9', '4'],
                  'C': ['c', 'c1', 'c2', 'c3'],
                  'D': ['d1', 'd1', 'd2', 'd3']})

and I want to update df1's C,D columns by df2 when they get same column values in A (if df1['A']==df2['A'] then df1['C']=df2['C'] and df1['D']=df2['D'])

the answer should be like this

      A  B   C   D  E
   0  1  1   c  d1  A
   1  2  1  A1  B1  A
   2  3  1  A2  B2  S
   3  4  1  c3  d3  S
   4  5  1  A4  B4  S

I tried df1.update(df2) but it just overwrite df1 by df2

>df1.update(df2)
>       A  B   C   D  E
     0  1  1   c  d1  A
     1  6  1  c1  d1  A
     2  9  1  c2  d2  S
     3  4  1  c3  d3  S
     4  5  1  A4  B4  S

and I tried pd.merge(df1, df2,how='inner' ,on=['A']) still not what I want 

   A  B C_x D_x  E C_y D_y
0  1  1   A  B0  A   c  d1
1  4  1  A3  B3  S  c3  d3

can anyone give me some suggestion? Thank you

Upvotes: 2

Views: 1188

Answers (1)

juanpa.arrivillaga
juanpa.arrivillaga

Reputation: 95873

I think this will be more space efficient:

Edit To Add

This may be more efficient:

In [22]: df1,df2 = df1.align(df2,join='left',axis=0)

In [23]: df1
Out[23]: 
   A  B   C   D  E
0  1  1   A  B0  A
1  2  1  A1  B1  A
2  3  1  A2  B2  S
3  4  1  A3  B3  S
4  5  1  A4  B4  S

In [24]: df2
Out[24]: 
     A    C    D
0    1    c   d1
1    6   c1   d1
2    9   c2   d2
3    4   c3   d3
4  NaN  NaN  NaN

Now you can do find a boolean array where the columns are equal, and use loc based assignment to modify df1 inplace without needed the extra columns:

In [26]: equal_rows = df1.A == df2.A

In [27]: df1.loc[equal_rows]
Out[27]: 
   A  B   C   D  E
0  1  1   A  B0  A
3  4  1  A3  B3  S

In [28]: df1.loc[equal_rows,['C','D']] = df2.loc[equal_rows,['C','D']]

In [29]: df1
Out[29]: 
   A  B   C   D  E
0  1  1   c  d1  A
1  2  1  A1  B1  A
2  3  1  A2  B2  S
3  4  1  c3  d3  S
4  5  1  A4  B4  S

And if you really need df2 as it was originally:

In [30]: df2.dropna(how='all',axis=0, inplace=True)

In [31]: df2
Out[31]: 
   A   C   D
0  1   c  d1
1  6  c1  d1
2  9  c2  d2
3  4  c3  d3

Original Answer

Here is a clunky way that is not space efficient:

In [13]: merged = pd.merge(df1,df2,how='left', on=['A'])

In [14]: merged
Out[14]: 
   A  B C_x D_x  E  C_y  D_y
0  1  1   A  B0  A    c   d1
1  2  1  A1  B1  A  NaN  NaN
2  3  1  A2  B2  S  NaN  NaN
3  4  1  A3  B3  S   c3   d3
4  5  1  A4  B4  S  NaN  NaN

In [15]: merged.fillna({'C_y':df1.C,'D_y':df1.D},inplace=True)
Out[15]: 
   A  B C_x D_x  E C_y D_y
0  1  1   A  B0  A   c  d1
1  2  1  A1  B1  A  A1  B1
2  3  1  A2  B2  S  A2  B2
3  4  1  A3  B3  S  c3  d3
4  5  1  A4  B4  S  A4  B4

In [16]: merged.drop(['C_x','D_x'],axis=1,inplace=True)

In [17]: merged
Out[17]: 
   A  B  E C_y D_y
0  1  1  A   c  d1
1  2  1  A  A1  B1
2  3  1  S  A2  B2
3  4  1  S  c3  d3
4  5  1  S  A4  B4

And if you want the original names:

In [20]: merged.rename(columns={"C_y":'C','D_y':'D'},inplace=True)

In [21]: merged
Out[21]: 
   A  B  E   C   D
0  1  1  A   c  d1
1  2  1  A  A1  B1
2  3  1  S  A2  B2
3  4  1  S  c3  d3
4  5  1  S  A4  B4

Upvotes: 2

Related Questions