CODEWITHSUNDEEP
pythonpandasdictionary
Rakesh Adhikesavan
Rakesh Adhikesavan

Reputation: 12826

How do I convert my DataFrame into a dictionary while ignoring the NaN values?

I have a DataFrame that has the following structure:

     Question   0      1          2        3       4   5   6   7   8  
0        1     NaN  Never  Sometimes  Usually  Always NaN NaN NaN NaN   
1        2     NaN  Never  Sometimes  Usually  Always NaN NaN NaN NaN   
2        3     NaN  Never  Sometimes  Usually  Always NaN NaN NaN NaN   
3        4     NaN  Never  Sometimes  Usually  Always NaN NaN NaN NaN

I want to convert this into a dictionary of dictionaries. The outer dictionary's key values are obtained from the column 'Question'. So I tried the following:

dictionary = lookup.set_index('Question').T.to_dict()

I got:

{1: {'0': nan,
  '1': 'Never',
  '2': 'Sometimes',
  '3': 'Usually',
  '4': 'Always',
  '5': nan,
  '6': nan,
  '7': nan,
  '8': nan,
  '9': nan},
 2: {'0': nan,
  '1': 'Never',
  '2': 'Sometimes',
  '3': 'Usually',
  '4': 'Always',
  '5': nan,
  '6': nan,
  '7': nan,
  '8': nan,
  '9': nan} ......

How do I drop the keys with the NaN values? The desired dictionary is:

{1: {'1': 'Never',
  '2': 'Sometimes',
  '3': 'Usually',
  '4': 'Always'},
 2: {'1': 'Never',
  '2': 'Sometimes',
  '3': 'Usually',
  '4': 'Always'} ......

Upvotes: 2

Views: 1504

Answers (2)

Zeugma
Zeugma

Reputation: 32105

Loop on your rows and generate a dict per series (question) with na values dropped:

{q: s.dropna().to_dict() for q, s in df.set_index('Question').iterrows()}

Upvotes: 4

mdml
mdml

Reputation: 22902

You can use a dictionary comprehension, where you filter out the non-string values if they are NaNs (using numpy.isnan):

>>> from numpy import isnan, nan
>>> d = {1: {'0': nan,                                                 
...:         '1': 'Never',                  
...:         '2': 'Sometimes',
...:         '3': 'Usually',
...:         '4': 'Always',
...:         '5': nan,
...:         '6': nan}, 
...:     2: {'0': nan,
...:         '1': 'Never',
...:         '2': 'Sometimes',
...:         '3': 'Usually',
...:         '4': 'Always',
...:         '5': nan,
...:         '6': nan}}
>>> { k1: { k2: v for k2, v in d2.iteritems() if type(v) == str or not isnan(v) }
      for k1, d2 in d.iteritems() }
{1: {'1': 'Never', '2': 'Sometimes', '3': 'Usually', '4': 'Always'},
 2: {'1': 'Never', '2': 'Sometimes', '3': 'Usually', '4': 'Always'}}

Upvotes: 2

Related Questions