Instantiate a type that is a TypeVar

Question

As a C++ programmer the following code seems very natural to me but it doesn't run:

from typing import TypeVar, Generic, List, NewType

TPopMember = TypeVar('TPopMember')
Population = NewType('Population', List[TPopMember])
class EvolutionaryAlgorithm(Generic[TPopMember]):
    def __init__(self, populationSize: int) -> None:
        # The following raises TypeError: 'TypeVar' object is not callable
        self.__population = Population([TPopMember() for _ in range(populationSize)])

Apparently Python isn't able to instantiate classes (the TPopMember) that are actually TypeVars. I simply want to create a list (Population) with a couple of default initialized (how do you say that in Python?) TPopMembers. How should I go about this?

I'm using Python 3.7.2.

Answer 1

A much easy solution could be to use functools.partial in an enum factory:

import random
from typing import Any
from enum import Enum
from functools import partial

class PopMember:
    def __init__(self):
        self.x = random.randint(0, 100)
    def __repr__(self):
        return "Pop({})".format(self.x)

class PapMember:
    def __init__(self):
        self.x = random.randint(0, 200)
    def __repr__(self):
        return "Pap({})".format(self.x)

class EvolutionaryAlgorithm:
    def __init__(self, member_type: Any, populationSize: int) -> None:
        self.__member_type = member_type
        self.__population = [self.__member_type() for _ in  range(populationSize)]

    @property
    def population(self):
        return self.__population

class EvolutionaryAlgorithmFactory(Enum):
    POP_MEMBER = partial(EvolutionaryAlgorithm, PopMember)
    PAP_MEMBER = partial(EvolutionaryAlgorithm, PapMember)

    def __call__(self, *args, **kwargs) -> Any:
        return self.value(*args, **kwargs)

    def __str__(self) -> str:
        return self.name


evolution = EvolutionaryAlgorithmFactory.POP_MEMBER(100)
print(evolution.population)

Answer 2

There is another possibility that avoid problems when serializing classes (ie using pickle).

Instead of using Generic, you can do the following:

from typing import Callable, Any
import random
from enum import Enum
from functools import wraps

class PopMember:
    def __init__(self):
        self.x = random.randint(0, 100)
    def __repr__(self):
        return "Pop({})".format(self.x)

class PapMember:
    def __init__(self):
        self.x = random.randint(0, 200)
    def __repr__(self):
        return "Pop({})".format(self.x)


def check_type(func: Callable) -> Callable:
    """Decorator to check that the child class has defined the POINT_TYPE member attribute."""
    @wraps(func)
    def wrapper(obj, *args, **kwargs) -> Any:
        if not hasattr(obj, 'T'):
            raise NotImplementedError(
                "You can not instantiate an abstract class.")
        return func(obj, *args, **kwargs)
    return wrapper

class EvolutionaryAlgorithm:
    @check_type
    def __init__(self, populationSize: int) -> None:
        self.__population = [self.T() for _ in  range(populationSize)]

    @classmethod
    @check_type
    def create(cls, populationSize: int):
        """Example of classmethod."""
        # You can use T as cls.T
        return cls(populationSize=populationSize)

    @property
    def population(self):
        return self.__population

class EvolutionaryAlgorithmPopMember(EvolutionaryAlgorithm):
    T = PopMember

class EvolutionaryAlgorithmPapMember(EvolutionaryAlgorithm):
    T = PapMember

class EvolutionaryAlgorithmFactory(Enum):
    POP_MEMBER = EvolutionaryAlgorithmPopMember
    PAP_MEMBER = EvolutionaryAlgorithmPapMember

    def __call__(self, *args, **kwargs) -> Any:
        return self.value(*args, **kwargs)

    def __str__(self) -> str:
        return self.name


evolution = EvolutionaryAlgorithmFactory.POP_MEMBER(100)
print(evolution.population)

This will avoid a lot of problems, rather than hacking python internals.

The main advantage in here is that you can reuse the classmethod functions.

Answer 3

You can do the following:

from typing import TypeVar, Generic, List, NewType
import random

class PopMember:
    def __init__(self):
        self.x = random.randint(0, 100)
    def __repr__(self):
        return "Pop({})".format(self.x)

TPopMember = TypeVar('TPopMember')
Population = NewType('Population', List[TPopMember])
class EvolutionaryAlgorithm(Generic[TPopMember]):
    def __init__(self, populationSize: int) -> None:
        obj = self.__orig_class__.__args__[0]
        self.__population = Population([obj() for _ in  range(populationSize)])

    @property
    def population(self):
        return self.__population

evolution = EvolutionaryAlgorithm[PopMember](100)
print(evolution.population)

The type used to define the Generic class can be found, within the instance in here: self.__orig_class__.__args__[0].

For classmethods just use this -> cls.__args__[0]

Answer 4

You didn't realize that type hint is a hint. In other words, don't think it is a type at all. You can't instantiate them.

As I understand from your comment, your intention is doing what C++ template allows you to do. So here is my way to achieve that:

from typing import TypeVar, Generic, List, NewType, Type
import random

class PopMember:
    def __init__(self):
        self.x = random.randint(0, 100)
    def __repr__(self):
        return "Pop({})".format(self.x)

TPopMember = TypeVar("TPopMember")
Population = NewType('Population', List[TPopMember])

class EvolutionaryAlgorithm(Generic[TPopMember]):
    def __init__(self, member_class: Type[TPopMember], populationSize: int) -> None:
        self.__population = Population([member_class() for _ in range(populationSize)])
    def __repr__(self):
        return "EA({})".format(self.__population)

x = EvolutionaryAlgorithm(PopMember, 5)
print(x)

output:

EA([Pop(49), Pop(94), Pop(24), Pop(73), Pop(66)])

What you have to understand is that, if you derived a class from Generic[T], you need to use T some how when you create your class. In my example I create a dummy object and resolve its class and initiate it. Normally I would not write in this way, I can just throw in a class as parameber pass in a class to the constructor to request to generate items of this particular type because class itself, distinct from an instance of it, is also a Python object. (thanks chepner for the suggestion)