CODEWITHSUNDEEP
pythondictionary
avenger
avenger

Reputation: 141

Filter nested dictionary in python based on key values

How do I filter a nested dictionary in python based on key values:

d = {'data': {'country': 'US', 'city': 'New York', 'state': None},
     'tags': ['US', 'New York'],
     'type': 'country_info',
     'growth_rate': None
     }

I want to filter this dictionary to eliminate NoneType values so the resulting dict should be:

d = {'data': {'country': 'US', 'city': 'New York'},
     'tags': ['US', 'New York'],
     'type': 'country_info',
     }

Also, the dict can have multiple levels of nesting. I want to remove all NoneType values from the dict.

Upvotes: 3

Views: 4593

Answers (2)

Bruno Bronosky
Bruno Bronosky

Reputation: 70409

I really appreciate the answer by @Lattyware. It helped me filter out a nested object and remove empty values regardless of type being dict, list, or str.

Here is what I came up with:

remove-keys-with-empty-values.py

# remove-keys-with-empty-values.py
from pprint import pprint

def remove_keys_with_empty_values(item):
  if hasattr(item, 'items'):
    return {key: remove_keys_with_empty_values(value) for key, value in item.items() if value==0 or value}
  elif isinstance(item, list):
    return [remove_keys_with_empty_values(value) for value in item if value==0 or value]
  else:
    return item

d = {
     'string': 'value',
     'integer': 10,
     'float': 0.5,
     'zero': 0,
     'empty_list': [],
     'empty_dict': {},
     'empty_string': '',
     'none': None,
    }

d['nested_dict'] = d.copy()
l = d.values()
d['nested_list'] = l

pprint({
  "DICT FILTERED": remove_keys_with_empty_values(d),
  "DICT ORIGINAL": d,
  "LIST FILTERED": remove_keys_with_empty_values(l),
  "LIST ORIGINAL": l,
})

execution

python remove-keys-with-empty-values.py
    {'DICT FILTERED': {'float': 0.5,
                       'integer': 10,
                       'nested_dict': {'float': 0.5,
                                       'integer': 10,
                                       'string': 'value',
                                       'zero': 0},
                       'nested_list': [0,
                                       'value',
                                       10,
                                       0.5,
                                       {'float': 0.5,
                                        'integer': 10,
                                        'string': 'value',
                                        'zero': 0}],
                       'string': 'value',
                       'zero': 0},
     'DICT ORIGINAL': {'empty_dict': {},
                       'empty_list': [],
                       'empty_string': '',
                       'float': 0.5,
                       'integer': 10,
                       'nested_dict': {'empty_dict': {},
                                       'empty_list': [],
                                       'empty_string': '',
                                       'float': 0.5,
                                       'integer': 10,
                                       'none': None,
                                       'string': 'value',
                                       'zero': 0},
                       'nested_list': [{},
                                       0,
                                       'value',
                                       None,
                                       [],
                                       10,
                                       0.5,
                                       '',
                                       {'empty_dict': {},
                                        'empty_list': [],
                                        'empty_string': '',
                                        'float': 0.5,
                                        'integer': 10,
                                        'none': None,
                                        'string': 'value',
                                        'zero': 0}],
                       'none': None,
                       'string': 'value',
                       'zero': 0},
     'LIST FILTERED': [0,
                       'value',
                       10,
                       0.5,
                       {'float': 0.5,
                        'integer': 10,
                        'string': 'value',
                        'zero': 0}],
     'LIST ORIGINAL': [{},
                       0,
                       'value',
                       None,
                       [],
                       10,
                       0.5,
                       '',
                       {'empty_dict': {},
                        'empty_list': [],
                        'empty_string': '',
                        'float': 0.5,
                        'integer': 10,
                        'none': None,
                        'string': 'value',
                        'zero': 0}]}

Upvotes: 0

Gareth Latty
Gareth Latty

Reputation: 89057

You can define this recursively pretty easily with a dict comprehension.

def remove_keys_with_none_values(item):
    if not hasattr(item, 'items'):
        return item
    else:
        return {key: remove_keys_with_none_values(value) for key, value in item.items() if value is not None}

Recursion isn't too optimised in Python, but given the relatively small number of nestings that are likely, I wouldn't worry.

Looking before we leap isn't too Pythonic, I think it is a better option than catching the exception - as it's likely that the value will not be a dict most of the time (it is likely we have more leaves than branches).

Also note that in Python 2.x, you probably want to swap in iteritems() for items().

Upvotes: 10

Related Questions