CODEWITHSUNDEEP
pythonpandasdictionarydataframe
Walter U.
Walter U.

Reputation: 341

Extracting dictionary values from a pandas dataframe

I need to extra a features from a dataset I imported from a .json file.

This is what it looks like:

f1 = pd.read_json('https://raw.githubusercontent.com/ansymo/msr2013-bug_dataset/master/data/v02/eclipse/short_desc.json')

print(f1.head())


                                               short_desc
1       [{'when': 1002742486, 'what': 'Usability issue...
10      [{'when': 1002742495, 'what': 'API - VCM event...
100     [{'when': 1002742586, 'what': 'Would like a wa...
10000   [{'when': 1014113227, 'what': 'getter/setter c...
100001  [{'when': 1118743999, 'what': 'Create Help Ind...

In essence, I need to take 'short_desc' as the column name, and populate it with the string values directly below it: 'Usability issue...

So far, I've tried the following:

f1['desc'] = pd.DataFrame([x for x in f1['short_desc']])

Wrong number of items passed 19, placement implies 1

Is there an easy way to accomplish this without the use of loops? Could someone point this newbie in the right direction?

Upvotes: 3

Views: 4728

Answers (2)

BENY
BENY

Reputation: 323226

or you can try apply(PS: apply consider as a time cost function)

f1['short_desc'].apply(pd.Series)[0].apply(pd.Series)

Out[864]: 
                                                     what        when   who
1         Usability issue with external editors (1GE6IRL)  1002742486    21
10                 API - VCM event notification (1G8G6RR)  1002742495    10
100     Would like a way to take a write lock on a tea...  1002742586    24
10000   getter/setter code generation drops "F" in ".....  1014113227   331
100001  Create Help Index Fails with seemingly incorre...  1118743999  9571

Upvotes: 2

cs95
cs95

Reputation: 402313

Don't initialise a dataframe and try to assign it to a column - columns are meant to be pd.Series.

You should just assign the list comprehension directly, like this:

f1['desc'] = [x[0]['what'] for x in f1['short_desc']]

As an alternative, I would propose a solution not involving any lambda functions, using operator and pd.Series.apply:

import operator

f1['desc'] = f1.short_desc.apply(operator.itemgetter(0))\
                             .apply(operator.itemgetter('what'))
print(f1.desc.head())

1           Usability issue with external editors (1GE6IRL)
10                   API - VCM event notification (1G8G6RR)
100       Would like a way to take a write lock on a tea...
10000     getter/setter code generation drops "F" in ".....
100001    Create Help Index Fails with seemingly incorre...
Name: desc, dtype: object

Upvotes: 5

Related Questions