CODEWITHSUNDEEP
pandasdataframepython-3.6hierarchical
BoBoMann
BoBoMann

Reputation: 101

Creating new columns based on value from another column in pandas

I have this pandas dataframe with column "Code" that contains the sequential hierarchical code. My goal is to create new columns with each hierarchical level code and its name as followed:

Original data:

    Code    Name
0   A       USA
1   AM      Massachusetts
2   AMB     Boston
3   AMS     Springfield
4   D       Germany
5   DB      Brandenburg
6   DBB     Berlin
7   DBD     Dresden

My Goal:

Code    Name           Level1   Level1Name      Level2  Level2Name      Level3      Level3Name
0   A   USA             A           USA          AM     Massachusetts   AMB         Boston
1   AM  Massachusetts   A           USA          AM     Massachusetts   AMB         Boston
2   AMB Boston          A           USA          AM     Massachusetts   AMB         Boston
3   AMS Springfield     A           USA          AM     Massachusetts   AMS         Springfiled
4   D   Germany         D           Germany      DB     Brandenburg     DBB         Berlin
5   DB  Brandenburg     D           Germany      DB     Brandenburg     DBB         Berlin
6   DBB Berlin          D           Germany      DB     Brandenburg     DBB         Berlin
7   DBD Dresden         D           Germany      DB     Brandenburg     DBD         Dresden

My Code: 

import pandas as pd
df = pd.read_excel(r'/Users/BoBoMann/Desktop/Sequence.xlsx')
df['Length']=test.Code.str.len() ## create a column with length of each cell in Code
df['Level1']=test.Code.str[:1]   ## create the first level using string indexing
df['Level1Name'] = df[df['Length']==1]['Name']
df.head() ## This yields:



Code    Name          Length    Level1  Level1Name
0   A       USA             1         A     USA
1   AM      Massachusetts   2         A     NaN
2   AMB     Boston          3         A     NaN
3   AMS     Springfield     3         A     NaN
4   D       Germany         1         D     Germany
5   DB      Brandenburg     2         D     NaN
6   DBB     Berlin          3         D     NaN
7   DBD     Dresden         3         D     NaN

For my current approach, how do I turn those NaN into USA and Germany respectively in Level1Name column?

Generally, is there a better approach to reach my goal of creating columns for each hierarchical layer and match them with their respective name in another column?

Upvotes: 3

Views: 2876

Answers (1)

Scott Boston
Scott Boston

Reputation: 153560

IIUC, let's use this code:

df['Codes'] = [[*i] for i in df['Code']]
df_level = df['Code'].str.extractall('(.)')[0].unstack('match').bfill().cumsum(axis=1)
s_map = df.explode('Codes').drop_duplicates('Code', keep='last').set_index('Code')['Name']
df_level.columns = [f'Level{i+1}' for i in df_level.columns]
df_level_names =  pd.concat([df_level[i].map(s_map) for i in df_level.columns], 
                            axis=1, 
                            keys=df_level.columns+'Name')
df_out = df.join([df_level, df_level_names]).drop('Codes', axis=1)
df_out

Output:

  Code           Name Level1 Level2 Level3 Level1Name     Level2Name   Level3Name
0    A            USA      A     AM    AMB        USA  Massachusetts       Boston
1   AM  Massachusetts      A     AM    AMB        USA  Massachusetts       Boston
2  AMB         Boston      A     AM    AMB        USA  Massachusetts       Boston
3  AMS    Springfield      A     AM    AMS        USA  Massachusetts  Springfield
4    D        Germany      D     DB    DBB    Germany    Brandenburg       Berlin
5   DB    Brandenburg      D     DB    DBB    Germany    Brandenburg       Berlin
6  DBB         Berlin      D     DB    DBB    Germany    Brandenburg       Berlin
7  DBD        Dresden      D     DB    DBD    Germany    Brandenburg      Dresden

Explained:

  • Unpack string into a list of characters creating 'Codes' column
  • Create 'LevelX' columns using extractall and regex . to get a single character, then bfill NaN above and cumsum along rows to create 'LevelX' columns
  • Create a pd.Series to use with map by calling explode on 'Codes' column create above and drop_duplicates keep the last value of 'Code' and then set_index on 'Codes' and keep 'Name' column to create 's_map'.
  • Rename name df_level columns to get Level1 instead of Level0.
  • Use pd.concat with list comprehension to map df_level columns to df_level_names using s_map. Also, using keys parameter to rename new columns and appending 'Name'
  • Use join to join df with df_levels and df_level_names, then drop the 'Codes' column, creating the desired output.

Upvotes: 2

Related Questions