CODEWITHSUNDEEP
pythonpandasdataframe
Rory LM
Rory LM

Reputation: 180

Pandas DataFrame efficiently split one column into multiple

I have a dataframe similar to this:

data = {"col_1": [0, 1, 2],
        "col_2": ["abc", "defg", "hi"]}
df = pd.DataFrame(data)

Visually:

   col_1 col_2
0      0   abc
1      1   defg
2      2   hi

What I'd like to do is split up each character in col_2, and append it as a new column to the dataframe

example iterative method:

def get_chars(string):
    chars = []
    for char in string:
        chars.append(char)
    return chars

char_df = pd.DataFrame()
for i in range(len(df)):
    char_arr = get_chars(df.loc[i, "col_2"])
    temp_df = pd.DataFrame(char_arr).T
    char_df = pd.concat([char_df, temp_df], ignore_index=True, axis=0)

df = pd.concat([df, char_df], ignore_index=True, axis=1)

Which results in the correct form:

   0     1  2  3    4    5
0  0   abc  a  b    c  NaN
1  1  defg  d  e    f    g
2  2    hi  h  i  NaN  NaN

But I believe iterating though the dataframe like this is very inefficient, so I want to find a faster (ideally vectorised) solution.

In reality, I'm not really splitting up strings, but the point of this question is to find a way to efficiently process one column, and return many.

Upvotes: 1

Views: 161

Answers (1)

jezrael
jezrael

Reputation: 862831

If need performance use DataFrame constructor with convert values to lists:

df = df.join(pd.DataFrame([list(x) for x in df['col_2']], index=df.index))

Or:

df = df.join(pd.DataFrame(df['col_2'].apply(list).tolist(), index=df.index))

print (df)
   col_1 col_2  0  1     2     3
0      0   abc  a  b     c  None
1      1  defg  d  e     f     g
2      2    hi  h  i  None  None

Upvotes: 3

Related Questions