How to access the type arguments of typing.Generic?

Question

The typing module provides a base class for generic type hints: The typing.Generic class.

Subclasses of Generic accept type arguments in square brackets, for example:

list_of_ints = typing.List[int]
str_to_bool_dict = typing.Dict[str, bool]

---

My question is, how can I access these type arguments?

That is, given str_to_bool_dict as input, how can I get str and bool as output?

Basically I'm looking for a function such that

>>> magic_function(str_to_bool_dict)
(<class 'str'>, <class 'bool'>)

Answer 1

Python >= 3.8

As of Python3.8 there is typing.get_args:

print( get_args( List[int] ) ) # (<class 'int'>,)

PEP-560 also provides __orig_bases__[n], which allows us the arguments of the nth generic base:

from typing import TypeVar, Generic, get_args

T = TypeVar( "T" )

class Base( Generic[T] ):
    pass

class Derived( Base[int] ):
    pass

print( get_args( Derived.__orig_bases__[0] ) ) # (<class 'int'>,)

---

Python >= 3.6

As of Python 3.6. there is a public __args__ and (__parameters__) field. For instance:

print( typing.List[int].__args__ )

This contains the generic parameters (i.e. int), whilst __parameters__ contains the generic itself (i.e. ~T).

---

Python < 3.6

Use typing_inspect.getargs

---

Some considerations

typing follows PEP8. Both PEP8 and typing are coauthored by Guido van Rossum. A double leading and trailing underscore is defined in as: "“magic” objects or attributes that live in user-controlled namespaces".

The dunders are also commented in-line; from the official repository for typing we can see:

• "__args__ is a tuple of all arguments used in subscripting, e.g., Dict[T, int].__args__ == (T, int)".

However, the authors also note:

• "The typing module has provisional status, so it is not covered by the high standards of backward compatibility (although we try to keep it as much as possible), this is especially true for (yet undocumented) dunder attributes like __union_params__. If you want to work with typing types in runtime context, then you may be interested in the typing_inspect project (part of which may end up in typing later)."

I general, whatever you do with typing will need to be kept up-to-date for the time being. If you need forward compatible changes, I'd recommend writing your own annotation classes.

Answer 2

The question asks specifically about typing.Generic, but it turns out that (at least in earlier versions of the typing module) not all subscriptable types are subclasses of Generic. In newer versions, all subscriptable types store their arguments in the __args__ attribute:

>>> List[int].__args__
(<class 'int'>,)
>>> Tuple[int, str].__args__
(<class 'int'>, <class 'str'>)

In python 3.5, however, some classes like typing.Tuple, typing.Union and typing.Callable stored them in different attributes like __tuple_params__, __union_params__ or generally in __parameters__. For completeness's sake, here's a function that can extract the type arguments from any subscriptable type in any python version:

import typing


if hasattr(typing, '_GenericAlias'):
    # python 3.7
    def _get_base_generic(cls):
        # subclasses of Generic will have their _name set to None, but
        # their __origin__ will point to the base generic
        if cls._name is None:
            return cls.__origin__
        else:
            return getattr(typing, cls._name)
else:
    # python <3.7
    def _get_base_generic(cls):
        try:
            return cls.__origin__
        except AttributeError:
            pass

        name = type(cls).__name__
        if not name.endswith('Meta'):
            raise NotImplementedError("Cannot determine base of {}".format(cls))

        name = name[:-4]
        try:
            return getattr(typing, name)
        except AttributeError:
            raise NotImplementedError("Cannot determine base of {}".format(cls))


if hasattr(typing.List, '__args__'):
    # python 3.6+
    def _get_subtypes(cls):
        subtypes = cls.__args__

        if _get_base_generic(cls) is typing.Callable:
            if len(subtypes) != 2 or subtypes[0] is not ...:
                subtypes = (subtypes[:-1], subtypes[-1])

        return subtypes
else:
    # python 3.5
    def _get_subtypes(cls):
        if isinstance(cls, typing.CallableMeta):
            if cls.__args__ is None:
                return ()

            return cls.__args__, cls.__result__

        for name in ['__parameters__', '__union_params__', '__tuple_params__']:
            try:
                subtypes = getattr(cls, name)
                break
            except AttributeError:
                pass
        else:
            raise NotImplementedError("Cannot extract subtypes from {}".format(cls))

        subtypes = [typ for typ in subtypes if not isinstance(typ, typing.TypeVar)]
        return subtypes


def get_subtypes(cls):
    """
    Given a qualified generic (like List[int] or Tuple[str, bool]) as input, return
    a tuple of all the classes listed inside the square brackets.
    """
    return _get_subtypes(cls)

Demonstration:

>>> get_subtypes(List[int])
(<class 'int'>,)
>>> get_subtypes(Tuple[str, bool])
(<class 'str'>, <class 'bool'>)

Answer 3

As far as I know, there is no happy answer here.

What comes to mind is the __args__ undocumented attribute which stores this information:

list_of_ints.__args__
>>>(<class 'int'>,)

str_to_bool_dict.__args__
>>>(<class 'str'>, <class 'bool'>)

but there is no mention of it in the documentation of the typing module.

It is worth noting that it was very close to be mentioned in the documentation though:

Probably we should also discuss whether we need to document all keyword arguments for GenericMeta.__new__. There are tvars, args, origin, extra, and orig_bases. I think we could say something about first three (they correspond to __parameters__, __args__, and __origin__ and these are used by most things in typing).

But it did not quite make it:

I added GenericMeta to __all__ and added docstrings to GenericMeta and GenericMeta.__new__ following the discussion in the issue. I decided not to describe __origin__ and friends in docstrings. Instead, I just added a comment at the place where they are first used.

From there, you still have three non-mutually exclusive options:

• wait for the typing module to reach full maturity and hope these features will be documented soon

• join the Python ideas mailing list and see if enough support can be gathered to make these internals public / part of the API

• work in the meantime with the undocumented internals, making a gamble that there won't be changes to these or that the changes will be minor.

Note that the third point can hardly be avoided as even the API can be subject to changes:

The typing module has been included in the standard library on a provisional basis. New features might be added and API may change even between minor releases if deemed necessary by the core developers.

Answer 4

Use the .__args__ on your constructs. So the magic function you need is something like --

get_type_args = lambda genrc_type: getattr(genrc_type, '__args__')

---

My question is, how can I access these type arguments?

In situations like these -- how do I access ...

Use Python's powerful introspection features.

Even as a non-pro programmer I know I am trying to inspect stuff and dir is a function which is like IDE in terminal. So after

>>> import typing
>>> str_to_bool_dict = typing.Dict[str, bool]

I want to see if there's anything that does the magic you want so

>>> methods = dir(str_to_bool_dict)
>>> methods
['__abstractmethods__', '__args__', .....]

I see too much info, to see if I am correct I verify

>>> len(methods)
53
>>> len(dir(dict))
39

Now let us find the methods which were designed specifically for generic types

>>> set(methods).difference(set(dir(dict)))
{'__slots__', '__parameters__', '_abc_negative_cache_version', '__extra__',
'_abc_cache', '__args__', '_abc_negative_cache', '__origin__',
'__abstractmethods__', '__module__', '__next_in_mro__', '_abc_registry',
'__dict__', '__weakref__'}

among these, __parameters__, __extra__, __args__ and __origin__ sound helpful. __extra__ and __origin__ won't work without self so we are left with __parameters__ and __args__.

>>> str_to_bool_dict.__args__
(<class 'str'>, <class 'bool'>)

Hence the answer.

---

Introspection allows py.test's assert statements to make JUnit derived testing frameworks look obsolete. Even languages like JavaScript / Elm / Clojure don't have a straight-forward thingy like dir of Python. Python's naming convention allows you to discover the language without actually reading (grokking in some cases such as these) the documentations.

So hunt using introspection and read documentation/mailing-lists to confirm your findings.

P.S. To OP -- this method also answers your question What's the correct way to check if an object is a typing.Generic? use discovery if you can't commit to mailing list or are a busy developer - that's the way to do it in python.

Answer 5

It seems that this inner method will do the trick

typing.List[int]._subs_tree()

which returns the tuple:

(typing.List, <class 'int'>)

But this a private API, probably there is a better answer.