Mimicking compile time interfaces in python

Question

Summary

This question first uses Java examples to demonstrate what I am trying to achieve, then shows Python equivalents where I attempt to replicate the compile time behavior, but fail.

Java

In Java, the compiler immediately throws an error if derived classes don't implement functionality required by abstract base classes or interfaces:

Using Abstract Base Class

abstract class A {
  abstract int f();
}

class B extends A {
/*
  int f(){
    return 0;
  }  
*/
}

class Main {
  public static void main(String[] args) {
    System.out.println("done");
  }
}

Comment out function f in class B definition:

• javac test.java fails with B is not abstract and does not override abstract method

Uncomment function f in class B definition:

• javac test.java succeeds

Using Interface

interface A {
  int f();
}

class B implements A {
/*
  public int f() {
    return 0;
  } 
*/
}

class Main {
  public static void main(String[] args) {
    System.out.println("done");
  }
}

Comment out function f in class B definition:

• javac test.java fails with B is not abstract and does not override abstract method

Uncomment function f in class B definition:

• javac test.java succeeds

Python

Interfaces don't really exist in python, but they are supposed to be closely approximated by the Protocol class in the typing module. Abstract base classes do exist in the abc module.

What I've found, however, is that the python interpreter never complains about unimplemented behavior unless derived class B is instantiated.

Using Abstract Base Class

from abc import abstractmethod, ABC


class A(ABC):
    @abstractmethod
    def f(self) -> int: ...

class B(A): ...

if __name__== "__main__":
    #b = B()
    print("done")

The following occurs if I comment out line 10:

- `python test.py` succeeds
- `mypy --strict test.py` succeeds

And the following occurs if I uncomment line 10

- `python test.py` fails with `TypeError: Can't instantiate abstract class B with abstract method f`
- `mypy --strict test.py` fails with `Cannot instantiate abstract class "B" with abstract attribute "f"`

Using Protocol

from typing import Protocol
from abc import abstractmethod

class A(Protocol):
    @abstractmethod
    def f(self) -> int: ...

class B(A): ...

if __name__=="__main__":
  b = B()
  print("done")

Just like with an abstract base class, the following occurs if I comment out line 10:

- `python test.py` succeeds
- `mypy --strict test.py` succeeds

And the following occurs if I uncomment line 10

- `python test.py` fails with `TypeError: Can't instantiate abstract class B with abstract method f`
- `mypy --strict test.py` fails with `Cannot instantiate abstract class "B" with abstract attribute "f"`

Question

I am not always instantiating the necessary classes which implement protocols or extend abc's in python. So I would like to mimic "compile" time interfaces in python, preferrably with mypy.

Is this possible? If not, why not?

Answer 1

I believe the crucial difference is that in Java you explictly declare your class as abstract or, by omission, concrete. Then it is an error for a non-abstract class to inherit abstract methods from a base without concretely overriding them.

In Python a class implicitly is abstract if it has any abstract methods (whether inherited or defined in that class), and non-abstract if it doesn't. Therefore, it's not an error to define a class that fails to override its inherited abstract methods -- it just means you're defining an abstract derived class.

The @final decorator resolves the issue since a final abstract class makes no sense and is rejected. So, it asserts that the class is non-abstract, but as an incidental side-effect of its real purpose.

I don't have a full list of all the things mypy lets you do with a concrete class but not an abstract one, that could be used as a check. If we had such a list then we could exhaustively search for a solution, and if none of them helps then that proves there isn't one! For example I believe Type[X] used to be disallowed for abstract X, but I don't think it is any more.

If you don't mind some horrible boilerplate per concrete class, then after each non-final class that you want to assert is concrete, you could write:

@final
class concrete_check(B):
    pass

mypy will reject this if B is abstract.

I had a thought that it might be possible to do something like:

T = TypeVar("T", bound=type)
def concrete(clz: Type[T]):
    @final
    class Foo(But what goes here?):
        pass
    return clz

The problem of course is that Python wants you to put clz as the base class of Foo, or in general a class, and so will not accept T there. mypy requires you to supply a compile-time-resolvable type as the base class, so it won't accept a variable clz. And of course since Python won't accept T, mypy rejects it too.

As a last resort you could easily enough write a class decorator that examines all the methods of the class, and raises an exception if any of them is abstract, and then decorate your class with concrete. But that's only going to trigger when the decorator is executed at runtime, not at static analysis time.

Answer 2

I have found a workaround with mypy: the decorator @final. Example:

from typing import final

class A(ABC):   
    @abstractmethod
    def f(self) -> int: ...    
    @abstractmethod
    def g(self) -> int: ...

@final
class B(A):    
    def f(self) -> int: ...
    

will raise the error: Mypy: Final class B has abstract attributes "g"

The only problem is that a final class cannot be subclassed.