CODEWITHSUNDEEP
pythonpandaslistdataframepivot-table
enixon4
enixon4

Reputation: 51

Pivot Pandas Column of Lists

I have a pandas dataframe that has a column whose values are lists and where another column is a date. I would like to create a dataframe that counts the elements of the lists by date.

The dataframe looks like:

image of dataframe. I'm not yet awesome enough to post pictures directly

pd.DataFrame(
    data={
        "col1": ["['a','b']", "['b','c']", "['a','c']", "", "['b']"],
        "col2": ["2020-01-01", "2020-01-02", "2020-01-03", "2020-01-04", "2020-01-05"],
    },
    index=[0, 1, 2, 3, 4],
)

What I would like the dataframe to look like is:

Image of desired dataframe

pd.DataFrame(
    data={"a": [1, 0, 1, 0, 0], "b": [1, 1, 0, 0, 1], "c": [0, 1, 1, 0, 0]},
    index=["2020-01-01", "2020-01-02", "2020-01-03", "2020-01-04", "2020-01-05"],
)

Any thoughts on how to do this kind of transformation?

Upvotes: 1

Views: 728

Answers (3)

Serge de Gosson de Varennes
Serge de Gosson de Varennes

Reputation: 11504

You could do this this way:


df = pd.DataFrame(
    data={
        "col1": [['a','b'], ['b','c'], ['a','c'], ['c'], ['b']],
        "col2": ["2020-01-01", "2020-01-02", "2020-01-03", "2020-01-04", "2020-01-05"],
    }
)
df2 = df.explode('col1').reset_index(drop=True)
df2["value"]=1
pd.pivot_table(df2, values="value", index=["col2"], columns=["col1"], aggfunc=lambda x: 1, fill_value=0)

Upvotes: 2

Ch3steR
Ch3steR

Reputation: 20669

You can use pd.crosstab here.

df['col1'] = df['col1'].str.findall('\w+')
df_ = df.explode('col1')
pd.crosstab(df_['col2'], df_['col1']).reindex(df_['col2'].unique()).fillna(0)

col1          a    b    c
col2                     
2020-01-01  1.0  1.0  0.0
2020-01-02  0.0  1.0  1.0
2020-01-03  1.0  0.0  1.0
2020-01-04  0.0  0.0  0.0
2020-01-05  0.0  1.0  0.0

Upvotes: 2

Quang Hoang
Quang Hoang

Reputation: 150785

You can use extractall to extract the values inside '', then counts the values with groupby:

out= (df.col1.str.extractall("'([^']*)'")
   .groupby(level=0)[0].value_counts()
   .unstack(level=1,fill_value=0)
   .reindex(df.index, fill_value=0)
)

out.index= df['col2']
print(out)

Output:

0           a  b  c
col2               
2020-01-01  1  1  0
2020-01-02  0  1  1
2020-01-03  1  0  1
2020-01-04  0  0  0
2020-01-05  0  1  0

Upvotes: 2

Related Questions