CODEWITHSUNDEEP
pythonlistpandasdataframemerge
Kalpish Singhal
Kalpish Singhal

Reputation: 392

Pandas Merge row data with multiple values to Python list for a column

I have a data-frame that looks like

DATA

*id*,             *name*,                      *URL*,                 *Type*  
    2,             birth_france_by_region,    http://abc. com,       T1 
    2,             birth_france_by_region,    http://pt. python,     T2 
    3,             long_lat,                  http://abc. com,       T3 
    3,             long_lat,                  http://pqur. com,      T1 
    4,             random_time_series,        http://sadsdc. com,    T2 
    4,             random_time_series,        http://sadcadf. com,   T3
    5,             birth_names,               http://google. com,    T1 
    5,             birth_names,               http://helloworld. com,T2 
    5,             birth_names,               http://hu. com,        T3

I want a this dataframe to merge the rows where id are equal and have a list of Type corresponding list of URL so final output should be like

*id*, *name*,             *URL*,                               *Type*  
2,birth_france_by_region,  [http://abc .com,http://pt.python], [T1,T2] 
3,long_lat,           [http://abc .com,http://pqur. com],       [T3,T1] 
4,random_time_series, [http://sadsdc. com,http://sadcadf .com,],[T2,T3] 
5,birth_names,        [http://google .com,http://helloworld. com,
                                       http://hu. com] ,   [T1,T2,T3]

Upvotes: 13

Views: 19378

Answers (3)

Conor Lynch
Conor Lynch

Reputation: 1

The completed solution for the two columns given above is:

    df_new.groupby(['matching_value']).agg({
        'entity_id':lambda x: x.tolist(),
        'fullname': lambda x: x.tolist()}
                                           )

Upvotes: 0

jezrael
jezrael

Reputation: 862541

I think you need groupby and aggregate tuple and then convert to list:

df = df.groupby(['id','name']).agg(tuple).applymap(list).reset_index()

print (df)
   id                    name  \
0   2  birth_france_by_region   
1   3                long_lat   
2   4      random_time_series   
3   5             birth_names   

                                                 URL          Type  
0                 [http://abc.cm, http://pt.python]      [T1, T2]  
1                  [http://abc.cm, http://pqur.com]      [T3, T1]  
2            [http://sadsdc.com, http://sadcadf.com]      [T2, T3]  
3  [http://google.;com, http://helloworld.com, ht...  [T1, T2, T3]

Because in version 0.20.3 raise error:

df = df.groupby(['id','name']).agg(lambda x: x.tolist())

ValueError: Function does not reduce

Upvotes: 16

Laurent
Laurent

Reputation: 2023

This will give you the expected result for the "URL" column:

test.groupby(["id", "name"])['URL'].apply(list)

id  name                  
2   birth_france_by_region                 [http://abc. com, http://pt. python]
3   long_lat                                [http://abc. com, http://pqur. com]
4   random_time_series                [http://sadsdc. com, http://sadcadf. com]
5   birth_names               [http://google. com, http://helloworld. com, h...

However, I can't find a solution for both URL and Type columns.

I could propose to do it in 2 steps:

  • temp_table1 = test.groupby(["id", "name"])['URL'].apply(list)
  • temp_table2 = test.groupby(["id", "name"])['Type'].apply(list)
  • Merge temp_table1 & temp_table2

Upvotes: 1

Related Questions