Why doesn't Python have a "__req__" (reflected equality) method?

Question

I have a little helper class:

class AnyOf(object):
    def __init__(self, *args):
        self.elements = args
    def __eq__(self, other):
        return other in self.elements

This lets me do sweet magic like:

>>> arr = np.array([1,2,3,4,5])
>>> arr == AnyOf(2,3)
np.array([False, True, True, False, False])

without having to use a list comprehension (as in np.array(x in (2,3) for x in arr).

(I maintain a UI that lets (trusted) users type in arbitrary code, and a == AnyOf(1,2,3) is a lot more palatable than a list comprehension to the non-technically savvy user.)

However!

This only works one way! For example, if I were to do AnyOf(2,3) == arr then my AnyOf class's __eq__ method never gets called: instead, the NumPy array's __eq__ method gets called, which internally (I would presume) calls the __eq__ method of all its elements.

This lead me to wonder: why does Python not allow a right-sided equivalent to __eq__? (Roughly equivalent to methods like __radd__, __rmul__, et cetera.)

Answer 1

I wanted to know exactly what the CPython interpreter does when it encounters a == b.

import dis

def eq(a, b):
    return a == b

print(dis.dis(eq))

This gives you:

  1           0 LOAD_FAST                0 (a)
              2 LOAD_FAST                1 (b)
              4 COMPARE_OP               2 (==)
              6 RETURN_VALUE

So, it uses the COMPARE_OP op. Here is the code which handles this. (Btw, when looking at earlier versions of CPython, the eval frame code was much simpler and much easier to follow. Now it is heavily optimized now for speed. I'm looking at the current master here.)

You see that this uses the PyObject_RichCompare function. We find that here.

The relevant logic is this:

/* For Python 3.0.1 and later, the old three-way comparison has been
   completely removed in favour of rich comparisons.  PyObject_Compare() and
   PyObject_Cmp() are gone, and the builtin cmp function no longer exists.
   The old tp_compare slot has been renamed to tp_as_async, and should no
   longer be used.  Use tp_richcompare instead.

   See (*) below for practical amendments.

   tp_richcompare gets called with a first argument of the appropriate type
   and a second object of an arbitrary type.  We never do any kind of
   coercion.

   The tp_richcompare slot should return an object, as follows:

    NULL if an exception occurred
    NotImplemented if the requested comparison is not implemented
    any other false value if the requested comparison is false
    any other true value if the requested comparison is true

  The PyObject_RichCompare[Bool]() wrappers raise TypeError when they get
  NotImplemented.

  (*) Practical amendments:

  - If rich comparison returns NotImplemented, == and != are decided by
    comparing the object pointer (i.e. falling back to the base object
    implementation).

*/

/* Map rich comparison operators to their swapped version, e.g. LT <--> GT */
int _Py_SwappedOp[] = {Py_GT, Py_GE, Py_EQ, Py_NE, Py_LT, Py_LE};

static const char * const opstrings[] = {"<", "<=", "==", "!=", ">", ">="};

/* Perform a rich comparison, raising TypeError when the requested comparison
   operator is not supported. */
static PyObject *
do_richcompare(PyThreadState *tstate, PyObject *v, PyObject *w, int op)
{
    richcmpfunc f;
    PyObject *res;
    int checked_reverse_op = 0;

    if (!Py_IS_TYPE(v, Py_TYPE(w)) &&
        PyType_IsSubtype(Py_TYPE(w), Py_TYPE(v)) &&
        (f = Py_TYPE(w)->tp_richcompare) != NULL) {
        checked_reverse_op = 1;
        res = (*f)(w, v, _Py_SwappedOp[op]);
        if (res != Py_NotImplemented)
            return res;
        Py_DECREF(res);
    }
    if ((f = Py_TYPE(v)->tp_richcompare) != NULL) {
        res = (*f)(v, w, op);
        if (res != Py_NotImplemented)
            return res;
        Py_DECREF(res);
    }
    if (!checked_reverse_op && (f = Py_TYPE(w)->tp_richcompare) != NULL) {
        res = (*f)(w, v, _Py_SwappedOp[op]);
        if (res != Py_NotImplemented)
            return res;
        Py_DECREF(res);
    }
    /* If neither object implements it, provide a sensible default
       for == and !=, but raise an exception for ordering. */
    switch (op) {
    case Py_EQ:
        res = (v == w) ? Py_True : Py_False;
        break;
    case Py_NE:
        res = (v != w) ? Py_True : Py_False;
        break;
    default:
        _PyErr_Format(tstate, PyExc_TypeError,
                      "'%s' not supported between instances of '%.100s' and '%.100s'",
                      opstrings[op],
                      Py_TYPE(v)->tp_name,
                      Py_TYPE(w)->tp_name);
        return NULL;
    }
    return Py_NewRef(res);
}

That shows that you can indeed return NotImplemented in your __eq__ and how it is handled.

tp_richcompare could be overwritten by a custom type. But in the default case, it should be this. So as expected, usually this should call __eq__ etc.

Answer 2

An __req__ may have been considered more confusing than useful in the language. Consider if class Left defines __eq__ and class Right defines __req__, then Python is obliged to make a consistent decision about who gets called first in Left() == Right() (and we would presumably like the result to be equivalent, either way). They can't both win.

However, the Python datamodel does allow a way to do what you want here. The comparison can be controlled from either side of the operation, but you'll need to define AnyOf in a particular way. To control the eq from the right hand side of a comparison when the left side is an np.ndarray instance, AnyOf should be a subclass of np.ndarray.

if I were to do AnyOf(2,3) == arr then my AnyOf class's __eq__ method never gets called

Actually, no, there's a fundamental misunderstanding evident here. The left side always gets first try at handling the equality comparison, unless the right side type is a subclass of the left side type.

arr == AnyOf(2,3)

In the comparison shown above, your custom __eq__ is being called, because the numpy array calls it! So np.ndarray wins, and it decides to check once per element. It literally could do anything else, including not calling your AnyOf.__eq__ at all.

AnyOf(2,3) == arr

In the comparison shown above, your class does get the first try at the comparison, and it fails because of the way in was used - return other in self.elements is checking if an array is in a tuple.

Answer 3

The documentation about the __rxx__ methods like __radd__ states:

These functions are only called if the left operand does not support the corresponding operation and the operands are of different types.

While classes don't have __add__ or __sub__ methods per default, they do have __eq__:

>>> class A(object):
...     pass
>>> '__eq__' in dir(A)
True

This means __req__ would never be called unless you explicitly remove __eq__ from the other class or make __eq__ return NotImplemented.

You can solve your specific problem with np.in1d:

>>> np.in1d(arr, [2, 3])
array([False,  True,  True, False, False], dtype=bool)

Answer 4

This is the documentation on the data model:

There are no swapped-argument versions of these methods (to be used when the left argument does not support the operation but the right argument does); rather, __lt__() and __gt__() are each other’s reflection, __le__() and __ge__() are each other’s reflection, and __eq__() and __ne__() are their own reflection. If the operands are of different types, and right operand’s type is a direct or indirect subclass of the left operand’s type, the reflected method of the right operand has priority, otherwise the left operand’s method has priority. Virtual subclassing is not considered.

As stated in the comments above, what you want works, and __eq__ is essentially the sames as a potential __req__: it is called on the right hand side of == if the object on the left hand side returns NotImplemented:

In [1]: class A:
   ...:     def __eq__(self, other):
   ...:         return NotImplemented
   ...:     

In [2]: class B:
   ...:     def __eq__(self, other): 
   ...:         print("B comparing")
   ...:         return True
   ...:     

In [3]: B() == A()
B comparing
Out[3]: True

In [4]: A() == B()
B comparing
Out[4]: True

In [5]: A() == A()
Out[5]: False

As it comes, it even work with other, ordinary, objects:

In [10]: 5 == B()
B comparing
Out[10]: True

However, some objects may yield a TypeError on __eq__ instead of returning NotImplemented or False, and that makes this not reliable for all kinds of objects.

What happens in your case, is an incorrect use of the operator in with arrays and tuples inside your own __eq__ method. (Thanks @wim to have spotted this in another answer here).