kouign amann
kouign amann

Reputation: 11

Maintain explicit method signatures and LSP : dealing with extra arguments in Python

I'd like to share a OOP conception problem. My code is written in Python. Here is a first non-valid code snippet that illustrates my problem.

Snippet 1

class Model(ABC):
    @abstractmethod
    def f(self, x: float, *args: float) -> float: ...


class ModelA(Model):
    def f(self, x: float) -> float:
        # arbitrary operation to illustrate
        return x + 1


class ModelB(Model):
    def __init__(self, baseline: Model):
        self.baseline = baseline

    def f(self, x: float, covar: float, *args: float) -> float:
        # arbitrary operation to illustrate
        return x * covar + self.baseline.f(x, *args)


def one_future_function(model: Model, x, *args):
    return model.f(x, *args)


modela: Model = ModelA()
modelb: Model = ModelB(modela)

print(one_future_function(modela, 1))
print(one_future_function(modelb, 1, 2))

From a design perspective, I understand that this code does not adhere to type rules, specifically the Liskov Substitution Principle (LSP). Although Python allows it to run, I care about design principles and do not want to keep this code. With my current knowledge and my specifications (more details at the end), the only valid alternative I can think of in Python is the following, but I am not satisfied with it either.

Snippet 2


class Model(ABC):
    @abstractmethod
    def f(self, x: float, *args: float) -> float: ...


class ModelA(Model):
    def f(self, x: float, *args : float) -> float:
        return x + 1


class ModelB(Model):
    def __init__(self, baseline: Model):
        self.baseline = baseline

    def f(self, x: float, *args: float) -> float:
        covar = args[0]
        return x * covar + self.baseline.f(x, *args[1:])

Why am I not satisfied? Because now covar is hidden from the signature in ModelB, and in ModelA, there are still *args that are not used and won’t be expected in the signature. This would be confusing for users of ModelA objects.

Formally speaking, my problem is that sometimes the f method should have “extra” arguments like covar in Snippet 1. Keeping the signature explicit is important because future users of the software would expect f to have “covar” sometimes (and maybe other arguments) and sometimes only x. I’m hiding the details, but the main reason is to keep the code as close as possible to some mathematical formulas. Originally, the idea to rely on *args is to use the unpacking operator trick when Model object can composed of other Model object. Then, extra arguments are propagated and consumed by each baseline component.

I’m looking for another approach that satisfies typing rules and my desire to have explicit method signatures in case there are extra arguments. Additionally:

If you think I misunderstand some OOP principles, please explain what I’m missing and provide additional resources. I’m continuously learning.

Thanks in advance! :)

I also tried a alternative where I splitted Model operations in another class. Let's call it Function.

class Function(ABC):
    @abstractmethod
    def f(self, x: float) -> float: ...

    def add_args(*names : str):
        '''allows to add extra arguments treated like attributes'''


class FunctionA(Function):
    def f(self, x: float) -> float:
        # arbitrary operation to illustrate
        return x + 1


class FunctionB(Function):
    def __init__(self, baseline: Function):
        self.baseline = baseline
        self.add_args("covar")

    def f(self, x: float) -> float:
        # covar is now a attribute thanks to add_args method (not decribed here)
        # the value is set in Model
        return x * self.covar + self.baseline.f(x)

class Model:
    def __init__(self, function : Function):
        self.function = function

    def f(self, x : float, *args : float):
        # a setter not described here that returns error if args are expected in function
        self.function.args = args 
        return self.function.f(x)


modela = Model(FunctionA())
modelb = Model(FunctionB(FunctionA())

modela.f(1) # ok
modelb.f(1, 2) # ok
modelb.f(1) # Error : covar value is required

The problem with this alternative is that every time a contributor wants to add a new model, he would have to create a Function object and pass it to Model. That's not intuitive. In addition, covar does not exist in any method signature. I also pass on the details of add_args but it adds complicated lines of code relying and Python magic methods. It worked but it may be risky in terms of maintenance and harder to explain to other contributors.

Upvotes: 0

Views: 62

Answers (1)

kouign amann
kouign amann

Reputation: 11

TLDR

I think I've figured out what was the problem. I was fooled by type checkers because my type hinting was wrong. As a consequence, I was wrongly focusing on LSP. But here LSP simply does not apply.

Why LSP does not apply?

The definition of LSP is : if ModelA subtypes Model, what holds for Model-objects holds for ModelA-objects. But in this example, because of the unpacking operator, ModelA-objects cannot subtypes Model. So the premise “if ModelA subtypes Model” is wrong and LSP has nothing to do with that.

From what I know, the type of an object is the set of all its interface signatures. Here because of *args, the concrete methods signature of all the derived class will never be consistent to Model signatures. So the LSP will never apply here. On one hand, inheritance is only a mecanism that allows to reuse code between classes. So it does not necessarily guarantee that subtypes will be created. But on the other hand, in most cases, developpers must take care to subtype when using inheritance. For me, that's a big bias that was injected in automatic type checkers because subtyping does matter most of the time. But if type hinting is wrong, type checkers can orient their errors explanations towards the wrong direction. With *args like this example, the type hinting must be adapted and can't be as simple as it is in most cases.

Here, to demonstrate how type checkers are wrong because of incorrect type hinting, consider previous code snippet 2, valid for type checkers. Now because it pass type checking, one can be fooled and think that LSP could be applied. So ModelA and ModelB were both subtypes of Model. Now reconsider the above function :

def one_future_function(model: Model, x, *args):
    return model.f(x, *args)


modela: Model = ModelA()
modelb: Model = ModelB(modela)

one_future_function(modela)
one_future_function(modelb) # error

If the LSP could be valid, the last line could'nt throw an error because if one_future_function is valid for ModelA-objects, it must be valid for ModelB-objects. So code snippet 1 was not wrong because of LSP and in code snippet 2 LSP is still not valid.

In conclusion, I think the problem relies in my wrong code type hinting which makes type checker confused. If type hinting is wrong, then errors explanations could be wrong too. So I think type hinting must be done with care.

Type hinting solution

To correctly type hint *args, I think I need variadic generic type (PEP 646). Here is my solution.

from abc import ABC, abstractmethod
from typing import Generic, Tuple, TypeVarTuple

Ts = TypeVarTuple('Ts')


class Model(ABC, Generic[*Ts]):
    @abstractmethod
    def f(self, x: float, *args: *Ts) -> float: ...


class ModelA(Model[()]):
    def f(self, x: float) -> float:
        return x + 1


class ModelB(Model[*Tuple[float, *Ts]]):
    def __init__(self, baseline: Model[*Ts]):
        self.baseline = baseline

    def f(self, x: float, covar: float, *args: *Ts) -> float:
        return x * covar + self.baseline.f(x, *args)

Ts is a variadic generic type that allows to specify exactly the types of *args elements and their number in the tuple for each implementation of Model. Thanks to Generic, it allows to specify that :

  • concrete ModelA has empty tuple for *args
  • concrete ModelB has one tuple composed of a float in first position, followed by undefined TypeVarTuple varying depending on the baseline

Now one_future_function can be correctly type hint :

def one_future_function(model: Model[*Ts], x: float, *args: *Ts) -> float:
    return model.f(x, *args)


modela: ModelA = ModelA()
modelb: ModelB[()] = ModelB(modela)

Unfortunatly, the TypeVarTuple approach is quite new in Python and not all type checkers are adapted to it. With the above example, only mypy does not complain. Pylint and Pycharm integrated type checkers complain. For these last type checker, I think the only current solution is to manually deactivate them by adding config comments in the code.

Upvotes: 1

Related Questions