"TypeError: unhashable type" during lookup of unhashable object in dict/set

Question

Preface

I understand that dicts/sets should be created/updated with hashable objects only due to their implementation, so when this kind of code fails

>>> {{}}  # empty dict of empty dict
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: unhashable type: 'dict'

it's ok and I've seen tons of this kind of messages.

But if I want to check if some unhashable object is in set/dict

>>> {} in {}  # empty dict not in empty dict

I get error as well

Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: unhashable type: 'dict'

Problem

What is the rationale behind this behavior? I understand that lookup and updating may be logically connected (like in dict.setdefault method), but shouldn't it fail on modification step instead of lookup? Maybe I have some hashable "special" values that I handle in some way, but others (possibly unhashable) -- in another:

SPECIAL_CASES = frozenset(range(10)) | frozenset(range(100, 200))
...
def process_json(obj):
    if obj in SPECIAL_CASES:
        ...  # handle special cases
    else:
        ...  # do something else

so with given lookup behavior I'm forced to use one of the options

• LBYL way: check if obj is hashable and only after that check if it is one of SPECIAL_CASES (which is not great since it is based on SPECIAL_CASES structure and lookup mechanism restrictions, but can be encapsulated in separate predicate),

• EAFP way: use some sort of utility for "safe lookup" like

  def safe_contains(dict_or_set, obj):
      try:
          return obj in dict_or_set
      except TypeError:
          return False
  

• use list/tuple for SPECIAL_CASES (which is not O(1) on lookups).

Or am I missing something trivial?

Answer 1

I've found this issue on Python bug tracker. Long story short:

if

>>> set([1,2]) in {frozenset([1,2]): 'a'}

returned False it will be in some way counter-intuitive since values are equal

>>> set([1,2]) == frozenset([1,2])
True

So I think I'll write & use proper utilities where situation like this can possibly occur.

---

About the roots of the error: in CPython repo dict___contains__ function (which is a dict.__contains__ method implementation) calls PyObject_Hash function (which corresponds to hash function) -> for unhashable objects (like {} in our first case) calls PyObject_HashNotImplemented function -> generates this error.

Answer 2

As you have no doubt realized, sets and dicts are very similar in their inner workings. Basically the concept is that you have key - value pairs (or just keys with a set), and the key must never change (immutable). If an object were mutable, the hash would loose it's meaning as a unique identifier of the underlying data. If you can't tell if an object is unique or not, the meaning of a set of unique keys looses it's key property of uniqueness. This is why mutable types are disallowed in sets and as the keys of a dict. With your example: {} in {} # empty dict not in empty dict I think you have a slight misunderstanding, as dict.__contains__ only checks the keys of the dict, not the values. Since you can never have a dict as a key (because it's mutable) this is invalid.