CODEWITHSUNDEEP
pythonpython-3.xpandasdataframenan
Hardik Gupta
Hardik Gupta

Reputation: 4790

Concatenate two columns in pandas with NaN

I have a dataframe like this

df = (pd.DataFrame({'ID': ['ID1', 'ID2', 'ID3'], 
                        'colA': ['A', 'B', 'C'], 
                        'colB': ['D', np.nan, 'E']}))

df

    ID  colA   colB
0   ID1 A      D
1   ID2 B      NaN
2   ID3 C      E

I want to combine the two columns, however keep only column A if column B is NaN. Hence Expected output is

    ID  colA    colB    colC
0   ID1 A       D       A_D
1   ID2 B       NaN     B
2   ID3 C       E       C_E

Upvotes: 10

Views: 12537

Answers (3)

Pygirl
Pygirl

Reputation: 13349

Learned this from Datanovice's answer:

df['col_c'] = df[['colA', 'colB']].stack().groupby(level=0).agg('_'.join)
df

    ID  colA    colB    col_c
0   ID1 A       D       A_D
1   ID2 B       NaN     B
2   ID3 C       E       C_E

Upvotes: 7

bharatk
bharatk

Reputation: 4315

Using Series.str.cat() accessor.

  • sep='_' - Separator to be put between the two strings.
  • na_rep='' - To ignore NaN value, it's None or string value to replace in place of null values.
  • str.replace('_$', '')- To remove underscore at the end.

Ex.

import pandas as pd
import numpy as np

df = (pd.DataFrame({'ID': ['ID1', 'ID2', 'ID3'],
                        'colA': ['A', 'B', 'C'],
                        'colB': ['D', np.nan, 'E']}))


df['colC']= df.colA.str.cat(df.colB,sep="_",na_rep='').str.replace('_$', '')
print(df)

O/P:

    ID colA colB colC
0  ID1    A    D  A_D
1  ID2    B  NaN    B
2  ID3    C    E  C_E

Upvotes: 3

jezrael
jezrael

Reputation: 862661

Idea is add _ to second column with _, so after replace missing value by empty string is not added _ for missing values:

df['colC'] = df['colA'] + ('_' + df['colB']).fillna('')
print (df)
    ID colA colB colC
0  ID1    A    D  A_D
1  ID2    B  NaN    B
2  ID3    C    E  C_E

If not sure where are missing values (in colA or colB):

df['colC'] = (df['colA'].fillna('') + '_' + df['colB'].fillna('')).str.strip('_')

Also is possible test each column separately:

m1 = df['colA'].isna()
m2 = df['colB'].isna()

df['colC'] = np.select([m1, m2, m1 & m2], 
                        [df['colB'], df['colA'], np.nan], 
                        default=df['colA'] + '_' + df['colB'])
print (df)

    ID colA colB colC
0  ID1    A    D  A_D
1  ID2    B  NaN    B
2  ID3  NaN    E    E
3  ID4  NaN  NaN  NaN

Upvotes: 11

Related Questions