CODEWITHSUNDEEP
pythonpandasdataframepivot
Rusty Robot
Rusty Robot

Reputation: 1845

Pivot by group and merge in Pandas dataframe

I have two dataframes, company_df and car_df. A company can have multiple cars and a car can only have one company.

Company_DF

   Company_ID Company_Name
0           1         Ford
1           2       Holden
2           3          Kia

Car_DF

   Company_ID  Car_ID   Car_Name
0           1       1     Falcon
1           1       2      Focus
2           2       1  Commodore
3           3       1    Sorento
4           3       2        Rio
5           3       2   Sportage

The Rio and Sportage have the same Car_ID on purpose, about 1 percent of my rows have this issue, it is not something I can change in my data source.

I would like to pivot each group of cars, by company, so that the cars are all on one line. For example. 

   Company_ID Company_Name  Car_ID_1 Car_Name_1  Car_ID_2 Car_Name_2  Car_ID_3  Car_Name_3
0           1         Ford         1     Falcon       2        Focus       NaN         NaN
1           2       Holden         1  Commodore       NaN        NaN       NaN         NaN
2           3          Kia         1    Sorento         2        Rio         2    Sportage

What I have at the moment works for 99 of the rows, is slow, and a messy way of doing it. But I'm not sure how to improve on it. 

import pandas as pd
company_df = pd.DataFrame([[1, 'Ford'], [2, 'Holden'], [3, 'Kia']], columns=['Company_ID', 'Company_Name'])
car_df = pd.DataFrame([[1, 1, 'Falcon'], [1, 2, 'Focus'], [2, 1, 'Commodore'], [3, 1, 'Sorento'], [3, 2, 'Rio'], [3, 2, 'Sportage']], columns=['Company_ID', 'Car_ID', 'Car_Name'])
for i in range(1, 3): # looping through car ids up to maximum, I don't want to do this though
    car_by_id_df = car_df[car_df.Car_ID==i] # select cars with current loop iterator/index
    car_by_id_df.columns = map(lambda col: '{}_{}'.format(col, i), car_by_id_df.columns) # rename all columns with ID as suffix, 
    car_by_id_df.rename(columns={'Company_ID_{}'.format(i): 'Company_ID'}, inplace=True) # Rename joining column back to original
    company_df = company_df.merge(right=car_by_id_df, on='Company_ID', how='left') # Merge
print(company_df)

This returns the following. Note that Kia is duplicated because of Rio and Sportage have the same id. I can't change the data in the Car_ID column, and I'm not sure how else to pivot the dataframe. 

   Company_ID Company_Name  Car_ID_1 Car_Name_1  Car_ID_2 Car_Name_2
0           1         Ford         1     Falcon       2        Focus
1           2       Holden         1  Commodore       NaN        NaN
2           3          Kia         1    Sorento       2          Rio
3           3          Kia         1    Sorento       2     Sportage

How can I pivot my car_df by group and merge onto company_id?

Upvotes: 0

Views: 57

Answers (2)

Rusty Robot
Rusty Robot

Reputation: 1845

Found a solution. I don't like the use of the for loop but it does work, and relatively fast. 

import pandas as pd
Company_DF = pd.DataFrame([[1, 'Ford'], [2, 'Holden'], [3, 'Kia']], columns=['Company_ID', 'Company_Name'])
Car_DF = pd.DataFrame([[1, 1, 'Falcon'], [1, 2, 'Focus'], [2, 1, 'Commodore'], [3, 1, 'Sorento'], [3, 2, 'Rio'], [3, 2, 'Sportage']], columns=['Company_ID', 'Car_ID', 'Car_Name'])

Car_DF['rank'] = Car_DF.groupby(['Company_ID']).cumcount() + 1
for ranking_number in range(Car_DF['rank'].min(), Car_DF['rank'].max()):
    Ranked_Car_DF = Car_DF[Car_DF['rank']==ranking_number].copy()
    Ranked_Car_DF.columns = map(lambda col: '{}_{}'.format(col, ranking_number), Ranked_Car_DF.columns)
    Ranked_Car_DF.rename(columns={'Company_ID_{}'.format(ranking_number): 'Company_ID'}, inplace=True)
    Company_DF = Company_DF.merge(right=Ranked_Car_DF, on='Company_ID', how='left')
print(Company_DF)

Upvotes: 0

Georgina Skibinski
Georgina Skibinski

Reputation: 13387

This will do the trick:

res=Car_DF.set_index("Company_ID").stack().to_frame()

res["sub_no"]=res.groupby(level=[0,1]).cumcount().add(1).astype(str)

res=res.reset_index(level=1)
res["level_1"]=res["level_1"].str.cat(res["sub_no"], sep="_")

res=res.drop("sub_no", axis=1).set_index("level_1", append=True).unstack("level_1")
res.columns=map(lambda x: x[1], res.columns)
res=res[sorted(res.columns, key=lambda x: x.split("_")[-1])]
res=Company_DF.merge(res, on="Company_ID", how="left")

Outputs:

  Company_ID Company_Name  ... Car_ID_3 Car_Name_3
0          1         Ford  ...      NaN        NaN
1          2       Holden  ...      NaN        NaN
2          3          Kia  ...        2   Sportage

Upvotes: 1

Related Questions