Why can't subclasses of tuple and str support weak references in Python?

Question

As of version 3.8, Python's documentation on weak references states:

Several built-in types such as list and dict do not directly support weak references but can add support through subclassing.

CPython implementation detail: Other built-in types such as tuple and int do not support weak references even when subclassed.

It has been asked before which types support weak references and why the built-in types do not support weak references, but why do built-in types like tuple and int not support weak references in CPython?

Answer 1

The fact that built-in types do not support weak references (and also do not support dynamic attributes) is an implementation detail. Typically, weak references (and dynamic attributes) are not needed. Not supporting them allows the data structures for the built-in types to be smaller and operations on them to be more efficient. Since these types (especially tuples and dicts) are used throughout Python's internal implementation, it is worthwhile to use the more performant implementation.

That covers the built-in types, but what about subclasses?

The way weak references are currently implemented is that each class defines a fixed memory offset where each instance stores a reference to a container holding its weak references. This fixed offset can be specified for finite size objects like float because the instance's data is of fixed size and so the weak reference pointer can be put at a fixed offset after this data. Mutable objects like dict and list are stored in memory in two pieces. The first part contains the immutable data related to the instance, including a pointer to the second part which holds the mutable data (e.g. an array holding the list contents). Since the first part of the instance's data is fixed size, it is also possible to specify a fixed offset to the weak reference pointer for these mutable types. The problem with str and tuple is that they are both immutable and variably sized. Each instance stores all of its contents in its single block of memory. Thus it is not possible for the class to specify a fixed offset to a weak reference pointer that is valid for all instances. At first, I was puzzled why int could not be weak referenced like float. Then I discovered that int instances in Python take up a variable amount of memory depending on the value of the integer.

There are different ways that data structures could be organized that would allow weak references to work for built-in types or for subclasses of them, but so far the need has not been great enough to incur the performance impact or the disruption of such a fundamental change.

Note: this answer is based on two posts (here and here) made by Raymond Hettinger to the python-list mailing on March 30, 2005 with the subject line "Weakrefs to classes that derive from str"). I provide the extra detail about the date and subject because the python-list mailing list links seem to change over time but can be easily searched by date and subject.