Adding additional arguments to overridden functions without pylint complaining

Question

I would like to define a function f in a superclass that has a set of required positional arguments, and permit subclasses to provide their own versions of f that share the required arguments, but also have additional named arguments.

Further, I want to do this in such a way that mypy is able to check the types of everything, and pylint3 doesn't complain about anything.

Functions f and g below represent my best attempts thus far:

from typing import Any

class Base:
    def f(self, required_arg: int, *args: Any, **kwargs: Any) -> None:
        print(self, required_arg, args, kwargs)
    
    def g(self, required_arg: int, **kwargs: Any) -> None:
        print(self, required_arg, kwargs)

class A(Base):
    def f(self, required_arg: int, x: int = 42, *args: Any, **kwargs: Any) -> None:
        print(self, required_arg, x, args, kwargs)
    
    def g(self, required_arg: int, *, x: int = 42, **kwargs: Any) -> None:
        print(self, required_arg, x, kwargs)

This code works as expected (passing x=20 places it in kwargs for the base class, and x for the derived class), however running pylint3 yields the following warnings:

W: 11, 1: Keyword argument before variable positional arguments list in the definition of f function (keyword-arg-before-vararg)
W: 11, 1: Parameters differ from overridden 'f' method (arguments-differ)
W: 14, 1: Parameters differ from overridden 'g' method (arguments-differ)

There are, of course, a variety of ways to suppress that class of warning, either generally, by file, or by line, but I would like to declare the function in such a way that this is not necessary.

If it makes a difference, I don't actually care if the base class can access any of the additional arguments; kwargs is just part of the declaration so as to permit it to take arbitrary arguments. Additionally, it is acceptable to require that x be passed as a keyword argument, as long as it ends up in x and not kwargs['x'].

Answer 1

You can have parameters in the child class which don't feature in the parent class, so long as they are optional. This would be easiest without args and kwargs.

class Animal:
    def speak(self, volume: int) -> None:
        print(f"Animal speaking at volume {volume}")
        
class Parrot(Animal):
    def speak(self, volume: int, words: str ="Pretty Polly") -> None:
        print(f"Parrot says {words} at volume {volume}")
        
a = Animal()
a.speak(4)
p = Parrot()
p.speak(5) # fine, words takes the default as "Pretty Polly"
p.speak(6, "Bye") # Fine, words works from its position
p.speak(7, optional_arg="Yay") # Fine, words can be named

MyPy is happy with that. The basic rule you need to follow is that your child objects can be used as parent objects. In this case MyPy is happy is that a Parrot is still a valid Animal because it's still possible to call p.speak with just volume, like Animal requires.

You can do the same thing with a keyword only argument.

class Parrot(Animal):
    def speak(self, required_arg: int, *, optional_kwarg: str = "Hello again") -> None:
        print(f"A {required_arg} {optional_arg}")

p = P()
p.speak(8) # Fine, as required by base class Animal
p.speak(9, optional_kwarg="Bye again") # fine, as specially allowed of Parrots
p.speak(10, "Oops") # This one won't work, because there isn't a positional argument to put it in.

This is often all that you need. When you know that you have a Parrot, you can tell it what to say. If all you know is that you have some sort of Animal, because it might be a dog and dogs can't speak English [citation needed], you should restrict yourself to instructions that all animals can handle.

---

It is possible to get your base class using *args and **kwargs. You can use the positional only argument marker /. The cost of doing so is that you commit things to either being a positional argument or a keyword argument. The syntax goes something like this:

class Base:
    def f(self, required_pos_arg: int, /, *args, required_kwarg, **kwargs) -> None:
        print(f"Base {required_pos_arg} {required_kwarg} {args} {kwargs}")
        
class A(Base):
    def f(self, required_pos_arg: int, optional_pos_arg: float=1.2, /, *args, required_kwarg, optional_kwarg: str ="Hello", **kwargs) -> None:
        print(f"A {required_pos_arg} {optional_pos_arg} {optional_kwarg} {args} {kwargs}")

b = Base()
b.f(9, 0.2, "Additional_pos_arg", required_kwarg="bar", optional_kwarg="Foo", kwarg_member="Additional_kw_arg")

a = A()
a.f(9, 0.2, "Additional_pos_arg", required_kwarg="bar", optional_kwarg="Foo", kwarg_member="Additional_kw_arg")

By using the / marker at the end of the positional ones, and * (or *args) before the start of the keyword ones, you remove the ambiguity that comes from confusing positional and keyword arguments. You can add more positional arguments (again, they must be optional so that the bare bones call still works) to the end of list before *. You can add more keyword arguments (same restrictions) after the *. You'll note that the child class must also take args and kwargs because the child must accept anything the parent would. And then MyPy is happy again.

---

However, as I mentioned earlier, this is probably not what you want to do.

The fundamental purpose of using MyPy is to get the computer to help your ensure that you're only using things where they make sense. If you add *args and **kwargs you necessarily limit how good a job it can do. It's no longer able to stop you from asking the dog to recite Macbeth, or even to stop you from asking the Parrot to speak with florgle of 9.6.

Except in very specific circumstances where you really want the base class to handle everything, it is safer to restrict yourself to what you actually want to do.

You will of course then find that if you write this code

def EnglishTest(students: List[Animal]):
    for s in students:
        s.speak(5, words="Double, double toil and trouble")

MyPy will yell at you. That's not a bug. It's doing its job. It's telling you that most animals don't do words. And it prompts you to write instead

def EnglishTest(students: List[Animal]):
    for s in students:
        if isinstance(s, Parrot):
            s.speak(5, words="Double, double toil and trouble")
        else:
            print(f"{s} failed its English test.")