CODEWITHSUNDEEP
pythonloopspandas
Flow Nuwen
Flow Nuwen

Reputation: 547

Extract List from Pandas Cell and Use List Elements as new column

Trying to create a dataframe that has the name of the election race, result (Republican - Democratic Popular Vote, as a fraction), and polling difference per poll. My code so far:

def results_polls_diff(editinfo, polls):
    rows = []
    for i, election in enumerate(editinfo):
        polls_key = election['slug']
        this_election = polls[polls_key]

        npolls = this_election.shape[0]

        diff = (this_election[candidates['R'].ix[i]] - this_election[candidates['D or I'].ix[i]])/100


        for c in election['estimates']:
            if c['party'] == 'Rep' :
                r1 = c['value']

        for c in election['estimates']:
            if c['party'] == 'Dem' or c['party'] == 'ind' :
                r2 = c['value']

        result = (r1-r2)/100

        #init_rows = []
        #for d in diff:
        #    init_rows.append((polls_key, result, d))
        #return init_rows

        rows.append((polls_key, result, [d for d in diff]))
    return rows
result_df = pd.DataFrame(results_polls_diff(editinfo, polls), columns = ['race', 'result', 'diff_list'])
result_df.head()

output:

race                                   result       diff_list
0   2014-delaware-senate-wade-vs-coons  -0.220  [-0.18, -0.16, -0.25, -0.15]
1   2014-massachusetts-senate-herr-vs-markey    -0.207  [-0.2, -0.15, -0.16, -0.25, -0.22, -0.26, -0.2...
2   2014-rhode-island-senate-zaccaria-vs-reed   -0.207  [-0.45, -0.42, -0.35]
3   2014-montana-senate-daines-vs-curtis    0.177   [0.14, 0.18, 0.16, 0.21, 0.13]
4   2014-hawaii-senate-cavasso-vs-schatz    -0.477  [-0.52, -0.26, -0.51, -0.54, -0.37, -0.32]

What I'm aiming for is something more like this:

race                                  result    diff_list
0   2014-delaware-senate-wade-vs-coons  -0.22   -0.18
1   2014-delaware-senate-wade-vs-coons  -0.22   -0.16
2   2014-delaware-senate-wade-vs-coons  -0.22   -0.25
3   2014-delaware-senate-wade-vs-coons  -0.22   -0.15

If I used the hashed out part of my code and change the append to rows.append((init_rows)), I get that result, but it doesn't seem to iterate through all of editinfo anymore. So the solution I'm looking for is either a way to get the iteration to work, or extract a list from the diff_list column so that the element occupy a single cell in that column, and duplicate the rest of the row.

Upvotes: 1

Views: 4197

Answers (1)

piRSquared
piRSquared

Reputation: 294248

This is one strategy. Consider the df

df = pd.DataFrame(dict(A=list('ab'), B=[1, 2], C=[[1, 2, 3], [4, 5, 6]], ))
df

enter image description here

option 1
use set_index, apply, unstack

df.set_index(['A', 'B']).C.apply(pd.Series).stack().reset_index(['A', 'B'], name='C')

option 2
build new index and dataframe, then unstack

names = ['A', 'B']
idx = pd.MultiIndex.from_tuples(df[names].values.tolist(), names=names)
pd.DataFrame(df.C.tolist(), idx).stack().reset_index(names, name='C')

enter image description here

Upvotes: 3

Related Questions