CODEWITHSUNDEEP
pythonpandas
PieSquare
PieSquare

Reputation: 327

Join merge multiple dataframes

My data is like this:

Name    test1   test2    test3     Count
Emp1    X,Y      A       a1,a2      1
Emp2    X       A,B,C    a3         2
Emp3    Z        C       a4,a5,a6   3

To split test1 and test2 cells with multiple values to individual rows and merged them together.

    df2 =  df.test1.str.split(',').apply(pd.Series)
    df2.index =  df.set_index(['Name', 'Count']).index
    df2=df2.stack().reset_index(['Name', 'Count'])
    df3 = df.test2.str.split(',').apply(pd.Series)
    df3.index = df.set_index(['Name', 'Count']).index
    df3=df3.stack().reset_index(['Name', 'Count'])

    df2.merge(df3,on=['Name', 'Count'],how='outer')

The out of code is :

Out[132]: 
   Name  Count 0_x 0_y
0  Emp1      1   X   A
1  Emp1      1   Y   A
2  Emp2      2   X   A
3  Emp2      2   X   B
4  Emp2      2   X   C
5  Emp3      3   Z   C

Code to split Test3 with multiple values to individual rows

    df4.index = df.set_index(['Name', 'Count']).index
    df4=df4.stack().reset_index(['Name', 'Count'])

Can anyone help me, how to multi-join Test3 with test2 and test1 Like i merged Test1 and Test in above code?

Upvotes: 1

Views: 61

Answers (3)

piRSquared
piRSquared

Reputation: 294218

I like using a comprehension

pd.DataFrame([
    (T.Name, T.Count, t1, t2)
    for T in df.itertuples()
    for t1, t2 in product(T.test1.split(','), T.test2.split(','))
], columns=['Name', 'Count', '0_x', '0_y'])

   Name  Count 0_x 0_y
0  Emp1      1   X   A
1  Emp1      1   Y   A
2  Emp2      2   X   A
3  Emp2      2   X   B
4  Emp2      2   X   C
5  Emp3      3   Z   C

Upvotes: 1

rafaelc
rafaelc

Reputation: 59264

(Not sure I understood right, but) Folllowing this answer, you can

expand(expand(df.drop('test3', 1), 'test1', ','), 'test2')

or

expand_all(df.drop('test3', axis=1), cols=['test1', 'test2'], seps=[',', ','])

where both output

    Name    test1   test2   Count
0   Emp1    X   A   1
1   Emp1    Y   A   1
2   Emp2    X   A   2
3   Emp2    X   B   2
4   Emp2    X   C   2
5   Emp3    Z   C   3

detail: 

def expand(df, col, sep=','):
    r = df[col].str.split(sep)
    d = {c: df[c].values.repeat(r.str.len(), axis=0) for c in df.columns}
    d[col] = [i for sub in r for i in sub]
    return pd.DataFrame(d)

Upvotes: 2

BENY
BENY

Reputation: 323226

More like 

df1=df.stack().str.split(',').apply(pd.Series)
df1.stack().unstack(level=2).groupby(level=[0,1]).ffill().reset_index(level=[0,1])
Out[124]: 
   Name  Count test1 test2 test3
0  Emp1      1     X     A    a1
1  Emp1      1     Y     A    a2
0  Emp2      2     X     A    a3
1  Emp2      2     X     B    a3
2  Emp2      2     X     C    a3
0  Emp3      3     Z     C    a4
1  Emp3      3     Z     C    a5
2  Emp3      3     Z     C    a6

Upvotes: 2

Related Questions