CODEWITHSUNDEEP
pythonpython-typing
micseydel
micseydel

Reputation: 405

How to dynamically use Python's type hinting to indicate a return value to be the same as a parameter's type

I'm using Python to read JSON from disk, and I'm trying to make sure my type hints are correct downstream. For example, something like this:

from typing import List

def verify_contains_ints(items: List[object]) -> List[int]:
    for item in items:
        if not isinstance(item, int):
            raise TypeError("List contents must be all ints")
    return items

The problem I'm running into is that I don't want to write separate functions for int, bool, str, etc. Is there a way to dynamically specify the type I want to verify? What I'd like like to write is something like this:

from typing import List

def verify_contains_type(items: List[object], inner_type = object) -> List[inner_type]:
    for item in items:
        if not isinstance(item, inner_type):
            raise TypeError(f"List contents must be all {inner_type}s")
    return items

Is there a way to do this in the current state of type hinting?

Note: this is a simplified version of what I'm actually trying to do. The default inner_type might seem silly here, but it is important for my use case.

Upvotes: 7

Views: 6670

Answers (3)

schot
schot

Reputation: 11278

To add to the answer of juanpa.arrivillaga, but with supporting a default inner_type of object. The best way I can find is using a combination of typing.Union and typing.overload. I must confess this is quite verbose, but as least no functional changes to the code are required.

Solution

from typing import List, Type, TypeVar, Union, cast, overload, 

T = TypeVar('T')


@overload
def verify_contains_type(items: List[object], inner_type: Type[T]) -> List[T]:
    ...

@overload
def verify_contains_type(
    items: List[object], inner_type: Type[object] = object
) -> List[object]:
    ...

def verify_contains_type(
    items: List[object], inner_type: Union[Type[T], Type[object]] = object
) -> Union[List[T], List[object]]:
    for item in items:
        if not isinstance(item, inner_type):
            raise TypeError(f"List contents must be all {inner_type!r}")
    return cast(List[T], items)

After this:

mylist: List[object] = [1, 2, 3, 4]

# Revealed type is 'builtins.list[builtins.float*]'
my_floats = verify_contains_type(mylist, float)


# Revealed type is 'builtins.list[builtins.object]'
my_whatevers = verify_contains_type(mylist)

Explanation

When analyzing the usage of the function, the type checker will only look at the @overload function definitions, checking them in the specified order until a match is found. The type annotations in the actual function are not taken into account.

When analyzing the code inside the body of the function itself, the type check will only use the type annotation of the actual function and ignore the @overload definitions.

Additional information on @overload:

Upvotes: 0

Michael0x2a
Michael0x2a

Reputation: 64288

Yes, but only if you are ok with constructing a new list:

from typing import List, Type, TypeVar

T = TypeVar('T')

def verify_contains(items: List[object], inner_type: Type[T]) -> List[T]:
    # Mypy currently needs this hint to infer what 'clean_items' is
    # supposed to contain. Other type checkers may not.
    clean_items: List[T] = []
    for item in items:
        if not isinstance(item, inner_type):
            raise TypeError("List contents must be all ints")
        clean_items.append(item)
    return clean_items

If you're not familiar with what TypeVars are, they're a way to let you write generic code. See https://mypy.readthedocs.io/en/stable/generics.html for more details.

Type lets you specify that you want a class object, rather than an instance of a class. See https://www.python.org/dev/peps/pep-0484/#the-type-of-class-objects for more details.

The reason why we need to create a new list is because if we don't, you could introduce a bug to your code due to mutation. For example:

original: List[object] = [3, 2, 1]
all_ints = verify_contains(original, int)

# Legal, since strs are a kind of object
original.append("boom")

# ...but if verify_contains doesn't return a copy, this will
# print out [3, 2, 1, "boom"]!
print(all_ints)

If you're ok with ignoring this potential bug, use a cast as suggested by some of the other answers.

Yet another alternative approach might be to just use a library like Pydantic instead of writing this validation logic yourself.

This is personally the approach I would take: I can focus on just writing high-level schemas using PEP 484 types and let the library handle validation for me.

Upvotes: 1

juanpa.arrivillaga
juanpa.arrivillaga

Reputation: 96277

I believe you can use typing.cast here, which is slightly ugly. Note, it has no run-time effects, it simply returns what was passed into it, although it does incur a function-call overhead. But it tells the type-checker "this is now of this type". You should use a TypeVar to make it generic, then simply pass the type like you were trying to do, and annotate it with Type

from typing import List, TypeVar, Type, cast

T = TypeVar('T')
def verify_contains_type(items: List[object], inner_type: Type[T]) -> List[T]:
    for item in items:
        if not isinstance(item, inner_type):
            raise TypeError("List contents must be all ints")
    return cast(List[T], items)


mylist: List[object] = [1, 2, 3, 4]

safe_mylist: List[int] = verify_contains_type(mylist, int)
print(safe_mylist[0] + safe_mylist[1])

mypy is happy now:

(py38) juan$ mypy --version
mypy 0.750
(py38) juan$ mypy test_typing.py
Success: no issues found in 1 source file

Upvotes: 5

Related Questions