Reputation: 101
I have this pandas dataframe with column "Code" that contains the sequential hierarchical code. My goal is to create new columns with each hierarchical level code and its name as followed:
Original data:
Code Name
0 A USA
1 AM Massachusetts
2 AMB Boston
3 AMS Springfield
4 D Germany
5 DB Brandenburg
6 DBB Berlin
7 DBD Dresden
My Goal:
Code Name Level1 Level1Name Level2 Level2Name Level3 Level3Name
0 A USA A USA AM Massachusetts AMB Boston
1 AM Massachusetts A USA AM Massachusetts AMB Boston
2 AMB Boston A USA AM Massachusetts AMB Boston
3 AMS Springfield A USA AM Massachusetts AMS Springfiled
4 D Germany D Germany DB Brandenburg DBB Berlin
5 DB Brandenburg D Germany DB Brandenburg DBB Berlin
6 DBB Berlin D Germany DB Brandenburg DBB Berlin
7 DBD Dresden D Germany DB Brandenburg DBD Dresden
My Code:
import pandas as pd
df = pd.read_excel(r'/Users/BoBoMann/Desktop/Sequence.xlsx')
df['Length']=test.Code.str.len() ## create a column with length of each cell in Code
df['Level1']=test.Code.str[:1] ## create the first level using string indexing
df['Level1Name'] = df[df['Length']==1]['Name']
df.head() ## This yields:
Code Name Length Level1 Level1Name
0 A USA 1 A USA
1 AM Massachusetts 2 A NaN
2 AMB Boston 3 A NaN
3 AMS Springfield 3 A NaN
4 D Germany 1 D Germany
5 DB Brandenburg 2 D NaN
6 DBB Berlin 3 D NaN
7 DBD Dresden 3 D NaN
For my current approach, how do I turn those NaN into USA and Germany respectively in Level1Name column?
Generally, is there a better approach to reach my goal of creating columns for each hierarchical layer and match them with their respective name in another column?
Upvotes: 3
Views: 2876
Reputation: 153570
IIUC, let's use this code:
df['Codes'] = [[*i] for i in df['Code']]
df_level = df['Code'].str.extractall('(.)')[0].unstack('match').bfill().cumsum(axis=1)
s_map = df.explode('Codes').drop_duplicates('Code', keep='last').set_index('Code')['Name']
df_level.columns = [f'Level{i+1}' for i in df_level.columns]
df_level_names = pd.concat([df_level[i].map(s_map) for i in df_level.columns],
axis=1,
keys=df_level.columns+'Name')
df_out = df.join([df_level, df_level_names]).drop('Codes', axis=1)
df_out
Output:
Code Name Level1 Level2 Level3 Level1Name Level2Name Level3Name
0 A USA A AM AMB USA Massachusetts Boston
1 AM Massachusetts A AM AMB USA Massachusetts Boston
2 AMB Boston A AM AMB USA Massachusetts Boston
3 AMS Springfield A AM AMS USA Massachusetts Springfield
4 D Germany D DB DBB Germany Brandenburg Berlin
5 DB Brandenburg D DB DBB Germany Brandenburg Berlin
6 DBB Berlin D DB DBB Germany Brandenburg Berlin
7 DBD Dresden D DB DBD Germany Brandenburg Dresden
extractall
and regex .
to get a
single character, then bfill
NaN above and cumsum
along rows to
create 'LevelX' columnsmap
by calling explode
on 'Codes'
column create above and drop_duplicates
keep the last value of
'Code' and then set_index
on 'Codes' and keep 'Name' column to
create 's_map'.pd.concat
with list comprehension to map
df_level columns to
df_level_names using s_map. Also, using keys
parameter to rename
new columns and appending 'Name'join
to join df with df_levels and df_level_names, then drop
the 'Codes' column, creating the desired output.Upvotes: 2