CODEWITHSUNDEEP
pythonpython-2.7python-3.xpandas
khangaroth
khangaroth

Reputation: 85

How to concatenate Pandas Dataframe columns dynamically?

I have a dataframe df (see program below) whose column names and number are not fixed. However, there is a list ls which will have the list of columns of df that needs to be appended together. I tried 

df['combined'] = df[ls].apply(lambda x: '{}{}{}'.format(x[0], x[1], x[2]), axis=1)

but here I am assuming that the list ls has 3 elements which is hard coding and incorrect.What if the list has 10 elements.. I want to dynamically read the list and append the columns of the dataframe.

import pandas as pd

def main():
    df = pd.DataFrame({
        'col_1': [0, 1, 2, 3],
        'col_2': [4, 5, 6, 7],
        'col_3': [14, 15, 16, 19],
        'col_4': [22, 23, 24, 25],
        'col_5': [30, 31, 32, 33],
    })

    ls = ['col_1','col_4', 'col_3']
    df['combined'] = df[ls].apply(lambda x: '{}{}'.format(x[0], x[1]), axis=1)
    print(df)

    if __name__ == '__main__':
         main()

Upvotes: 1

Views: 3324

Answers (2)

Bharath M Shetty
Bharath M Shetty

Reputation: 30605

You can use cumulative sum over strings for this for more speed i.e 

df[ls].astype(str).cumsum(1).iloc[:,-1].values

Output : 

0    02214
1    12315
2    22416
3    32519
Name: combined, dtype: object

If you need to add space then first add ' ' then find sum i.e 

n = (df[ls].astype(str)+ ' ').sum(1)

0    0 22 14 
1    1 23 15 
2    2 24 16 
3    3 25 19 
dtype: object

Timings :

ndf = pd.concat([df]*10000)

%%timeit
ndf[ls].astype(str).cumsum(1).iloc[:,-1].values
1 loop, best of 3: 538 ms per loop

%%timeit
ndf[ls].astype(str).apply(''.join, axis=1)
1 loop, best of 3: 1.93 s per loop

Upvotes: 1

akuiper
akuiper

Reputation: 215077

You can use ''.join after converting the columns' data type to str:

df[ls].astype(str).apply(''.join, axis=1)

#0    02214
#1    12315
#2    22416
#3    32519
#dtype: object

Upvotes: 5

Related Questions