CODEWITHSUNDEEP
pythonprotocolspython-typingmypykeyword-argument
David Hadley
David Hadley

Reputation: 504

How to define a Python Protocol that is Callable with any number of keyword arguments of Any type?

How do I define a Python protocol for a type that is:

This is my attempt:

from typing import Any, Protocol, TypeVar

T = TypeVar("T", covariant=True)


class Operation(Protocol[T]):
    def __call__(self, **kwargs: Any) -> T:
        pass


# some example functions that should be a structural sub-type of "Operation[str]"
def sumint(*, x: Any, y: Any) -> str:
    return f"{x} + {y} = {x + y}"


def greet(*, name: Any = "World") -> str:
    return f"Hello {name}"


# an example function that takes an "Operation[str]" as an argument
def apply_operation(operation: Operation[str], **kwargs: Any) -> str:
    return operation(**kwargs)


if __name__ == "__main__":
    print(apply_operation(sumint, x=2, y=2))
    # prints: 2 + 2 = 4
    print(apply_operation(greet, name="Stack"))
    # prints: Hello Stack

However, mypy produces the error:

example.py:26: error: Argument 1 to "apply_operation" has incompatible type "Callable[[NamedArg(Any, 'x'), NamedArg(Any, 'y')], str]"; expected "Operation[str]"
example.py:28: error: Argument 1 to "apply_operation" has incompatible type "Callable[[DefaultNamedArg(Any, 'name')], str]"; expected "Operation[str]"
Found 2 errors in 1 file (checked 1 source file)

What am I doing wrong? How do I make mypy happy?

Upvotes: 5

Views: 4498

Answers (3)

David Hadley
David Hadley

Reputation: 504

Using Callable[..., T], as suggested in the other answers, solves the problem in the question.

If one still needs to implement a custom Protocol (for example if additional methods need to be added to the protocol) this can be done in the following way:

from typing import Any, Protocol, TypeVar

T = TypeVar("T", covariant=True)

class Operation(Protocol[T]):
    __call__: Callable[..., T]

See: How to combine a custom protocol with the Callable protocol?

Upvotes: 2

user2357112
user2357112

Reputation: 281151

You cannot define a protocol that fits your requirements, because it's fundamentally unsafe from a static typing perspective.

The problem here is that although you say an instance of Operation[T] should be callable "with any number of keyword arguments of any type", you seem to actually mean that there is some combination of keyword arguments it accepts. It doesn't just accept any keyword arguments.

If you could define an Operation[T] protocol with the characteristics you want, then your apply_operation

def apply_operation(operation: Operation[str], **kwargs: Any) -> str:
    return operation(**kwargs)

would still be a type error. operation(**kwargs) is unsafe, because there's no guarantee that the provided keyword arguments are arguments operation accepts. You could call

apply_operation(sumint, name="Stack")

and that would fit the apply_operation signature, but it would still be an unsafe call.

If you want to annotate this, your best bet is probably to use Callable[..., T], as suggested in Alex Waygood's answer. Specifying ... as the argument type list for Callable effectively disables type checking for the callable's arguments, much like annotating a variable with Any effectively disables type checking for that variable.

Remember that this disables the safety checks - there will be no warning if you do something like apply_operation(sumint, name="Stack").

Upvotes: 4

Alex Waygood
Alex Waygood

Reputation: 7559

I can't answer your question about exactly why MyPy isn't happy — but here's a different approach that MyPy does seem to be happy with:

from typing import Any, Callable, TypeVar

T = TypeVar("T", covariant=True)


Operation = Callable[..., T]


# some example functions that should be a structural sub-type of "Operation[str]"
def sumint(*, x: int = 1, y: int = 2) -> str:
    return f"{x} + {y} = {x + y}"


def greet(*, name: str = "World") -> str:
    return f"Hello {name}"


# an example function that takes an "Operation[str]" as an argument
def apply_operation(operation: Operation[str], **kwargs: Any) -> str:
    return operation(**kwargs)


if __name__ == "__main__":
    print(apply_operation(sumint, x=2, y=2))
    # prints: 2 + 2 = 4
    print(apply_operation(greet, name="Stack"))
    # prints: Hello Stack

Upvotes: 3

Related Questions