Nested annotate fields in Django REST Framework serializers

Question

I am trying to view nested annotate (aggregated/calculated) fields in Django REST Framework serializers. This would allow to work more cleanly with annotated fields. This post is similar to Aggregate (and other annotated) fields in Django Rest Framework serializers however I would like a similar technique to work nested. Below the methodology is visible on how this works without nesting and how it doesn't seem to work with nesting.

I know this could be achieved manually (with a Django View) or by using methods that overload the database which I am not interested in. But maybe there is a performant and elegant solution for this problem.

The following works (not nested)

Models

class IceCreamCompany(models.Model):
    name = models.CharField(max_length=255)


class IceCreamTruck(models.Model):
    company = models.ForeignKey('IceCreamCompany', related_name='trucks')
    capacity = models.IntegerField()


class IceCreamTruckDriver(models.Model):
    name = models.CharField(max_length=255)
    first_name = models.CharField(max_length=255)
    truck = models.ForeignKey('IceCreamTruck', related_name='drivers')

Serializers

class IceCreamTruckDriverSerializer(serializers.ModelSerializer):

    class Meta:
        model = IceCreamTruckDriver
        fields = ('name', 'first_name')


class IceCreamTruckSerializer(serializers.ModelSerializer):
    drivers = IceCreamTruckDriverSerializer(many=True, read_only=True)

    class Meta:
        model = IceCreamTruck
        fields = ('capacity', 'drivers')


class IceCreamCompanySerializer(serializers.ModelSerializer):
    trucks = IceCreamTruckSerializer(many=True, read_only=True)
    amount_of_trucks = serializers.IntegerField()

    class Meta:
        model = IceCreamCompany
        fields = ('name', 'trucks', 'amount_of_trucks')

Viewset

class IceCreamCompanyViewSet(viewsets.ModelViewSet):
    queryset = IceCreamCompany.objects.prefetch_related('trucks', 'trucks__drivers')\
                           .annotate(amount_of_trucks=Count('trucks'))\
                           .all()

    serializer_class = IceCreamCompanySerializer

Result

"results": [
        {
            "name": "Pete Ice Cream",
            "trucks": [
                {
                    "capacity": 35,
                    "drivers": [
                        {
                            "name": "Damian",
                            "first_name": "Ashley"
                        },
                        {
                            "name": "Wilfrid",
                            "first_name": "Lesley"
                        }
                    ]
                },
                {
                    "capacity": 30,
                    "drivers": [
                        {
                            "name": "Stevens",
                            "first_name": "Joseph"
                        }
                    ]
                },
                {
                    "capacity": 30,
                    "drivers": []
                }
            ],
            "amount_of_trucks": 3
        }
    ]

The following does not work (nested)

Same models

Serializers

class IceCreamTruckDriverSerializer(serializers.ModelSerializer):

    class Meta:
        model = IceCreamTruckDriver
        fields = ('name', 'first_name')


class IceCreamTruckSerializer(serializers.ModelSerializer):
    drivers = IceCreamTruckDriverSerializer(many=True, read_only=True)
    amount_of_drivers = serializers.IntegerField()

    class Meta:
        model = IceCreamTruck
        fields = ('capacity', 'drivers', 'amount_of_drivers')


class IceCreamCompanySerializer(serializers.ModelSerializer):
    trucks = IceCreamTruckSerializer(many=True, read_only=True)

    class Meta:
        model = IceCreamCompany
        fields = ('name', 'trucks')

Viewset

class IceCreamCompanyViewSet(viewsets.ModelViewSet):
    queryset = IceCreamCompany.objects.prefetch_related('trucks', 'trucks__drivers')\
                           .annotate(trucks__amount_of_drivers=Count('trucks__drivers'))\
                           .all()

    serializer_class = IceCreamCompanySerializer

Result

AttributeError at /ice/
Got AttributeError when attempting to get a value for field `amount_of_drivers` on serializer `IceCreamTruckSerializer`.
The serializer field might be named incorrectly and not match any attribute or key on the `IceCreamTruck` instance.
Original exception text was: 'IceCreamTruck' object has no attribute 'amount_of_drivers'.

Answer 1

In case you don't want to override the manager of the model (as suggested by Campi), since it would have impact on all queries from the given model, there is one more way to achieve this.

1. Define a function in IceCreamCompany model where the counting of drivers is done along with the prefetching of related objects

   class IceCreamCompany(models.Model):
       name = models.CharField(max_length=255)
   
       def get_annotated_trucks(self):
           return IceCreamTruck.objects.filter(company_id=self.id).annotate(
           amount_of_drivers=Count('drivers')).prefetch_related('drivers')
   

2. Use the function to fetch IceCreamTruck objects in IceCreamCompanySerializer by specifying the function in the source parameter of the nested serializer

   class IceCreamCompanySerializer(serializers.ModelSerializer):
       trucks = IceCreamTruckSerializer(many=True, read_only=True, 
       source='get_annotated_trucks')
   
       class Meta:
           model = IceCreamCompany
           fields = ('name', 'trucks')
   

3. There is no need to do any action in the viewset (prefetching/annotating)

   class IceCreamCompanyViewSet(viewsets.ModelViewSet):
       queryset = IceCreamCompany.objects.all()
       serializer_class = IceCreamCompanySerializer
   

Note: this approach produces the same queries as the approach with custom manager

Answer 2

For reference, it is also possible to annotate the amount of drivers per Truck on the model IceCreamTruck, for example with a custom manager:

class AnnotatedManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().annotate(amount_of_drivers=Count('drivers'))

class IceCreamTruck(models.Model):
    company = models.ForeignKey('IceCreamCompany', related_name='trucks')
    capacity = models.IntegerField()

    objects = AnnotatedManager()

Then you don't need to annotate the viewset because amount_of_drivers is already annotated on trucks:

class IceCreamCompanyViewSet(viewsets.ModelViewSet):
    queryset = IceCreamCompany.objects.prefetch_related('trucks', 'trucks__drivers').all()
    serializer_class = IceCreamCompanySerializer 

It should be more efficient than counting inside the serializer.

Answer 3

I got an answer using the Django REST google groups to use read_only=True inside the IntegerField, which helped removing the error but then the field wasn't displayed anymore. Maybe my annotation was wrong. Anyway I ended up using a custom view in Django since I ended up needing more data. However you can get the data in other ways:

A very elegant solution would be to remove the annotate function and use a SerializerMethodField which can give me my result.

HOWEVER: this does make a lot of queries!!

Same models

Serializers

class IceCreamTruckDriverSerializer(serializers.ModelSerializer):

    class Meta:
        model = IceCreamTruckDriver
        fields = ('name', 'first_name')


class IceCreamTruckSerializer(serializers.ModelSerializer):
    drivers = IceCreamTruckDriverSerializer(many=True, read_only=True)
    amount_of_drivers = serializers.SerializerMethodField()

    def get_amount_of_drivers(self, obj):
        return obj.drivers.count()

    class Meta:
        model = IceCreamTruck
        fields = ('capacity', 'drivers', 'amount_of_drivers')


class IceCreamCompanySerializer(serializers.ModelSerializer):
    trucks = IceCreamTruckSerializer(many=True, read_only=True)

    class Meta:
        model = IceCreamCompany
        fields = ('name', 'trucks')

Viewset

class IceCreamCompanyViewSet(viewsets.ModelViewSet):
    queryset = IceCreamCompany.objects.prefetch_related('trucks', 'trucks__drivers').all()

    serializer_class = IceCreamCompanySerializer

Result

"results": [
        {
            "name": "Pete Ice Cream",
            "trucks": [
                {
                    "capacity": 35,
                    "drivers": [
                        {
                            "name": "Damian",
                            "first_name": "Ashley"
                        },
                        {
                            "name": "Wilfrid",
                            "first_name": "Lesley"
                        }
                    ],
                    "amount_of_drivers": 2
                },
                {
                    "capacity": 30,
                    "drivers": [
                        {
                            "name": "Stevens",
                            "first_name": "Joseph"
                        }
                    ],
                    "amount_of_drivers": 1
                },
                {
                    "capacity": 30,
                    "drivers": [],
                    "amount_of_drivers": 0
                }
            ]
        }
    ]

It's also possible to use functions inside the models like this: Django Rest Framework Ordering on a SerializerMethodField (it's visible in the code itself) but I didn't choose it so I don't have to modify my models too much. This also makes too many queries.