CODEWITHSUNDEEP
python
dzieciou
dzieciou

Reputation: 4524

__len__ can't return big numbers

The following code:

class Container:

    def __len__(self):
        return 10**100

c = Container()
print(len(c))

returns

OverflowError: cannot fit 'int' into an index-sized integer

I've read this issue has been marked as WON'T FIX but perhaps it was fixed in another issue? Any workaround besides using custom attribute for storing container size?

I'm using Python 3.6.9 and it has not been fixed in this version.

Upvotes: 0

Views: 309

Answers (1)

MisterMiyagi
MisterMiyagi

Reputation: 52019

This has not changed in any way and there is no workaround. It is a documented CPython implementation detail.

object.__len__(self)

Called to implement the built-in function len(). Should return the length of the object, an integer >= 0. Also, an object that doesn’t define a __bool__() method and whose __len__() method returns zero is considered to be false in a Boolean context.

CPython implementation detail: In CPython, the length is required to be at most sys.maxsize. If the length is larger than sys.maxsize some features (such as len()) may raise OverflowError. To prevent raising OverflowError by truth value testing, an object must define a __bool__() method.

Note that the maximum value is 9223372036854775807 on a 64 bit machine.

sys.maxsize

An integer giving the maximum value a variable of type Py_ssize_t can take. It’s usually 2**31 - 1 on a 32-bit platform and 2**63 - 1 on a 64-bit platform.

The restriction is due to __len__ being a Python representation of several C-API calls to get container lengths. The C-API requires a type of Py_ssize_t for this, which has the observed restrictions.

Py_ssize_t PySequence_Length(PyObject *o)

Returns the number of objects in sequence o on success, and -1 on failure. This is equivalent to the Python expression len(o).

Since OverflowError is a documented behaviour, it is acceptable for objects that might be too large for __len__ to provide it anyway. For example, the builtin range can overflow on len:

>>> import sys
>>> len(range(sys.maxsize))
9223372036854775807
>>> len(range(sys.maxsize + 1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C ssize_t

If the precise length is needed, objects are free to implement a separate method to return the length without using len.

Upvotes: 5

Related Questions