Django Rest Framework - deserialization of a TaggableManager field

Question

I used django-taggit to add tags to my model. Django Version: 2.2.10, Python Version: 3.8.1

Now I'm trying to integrate tags with django rest-framework, e.g. CREATE/UPDATE/REMOVE model instances with/without tags.

My problem: I'm not able to create (via rest api) a new instance of my model with tags. I can GET model instances without problems.

My models.py:

from taggit.managers import TaggableManager

class Task(models.Model):
    name = models.CharField(max_length=100, blank=False)
    ...
    tags = TaggableManager(blank=True)

    def get_tags(self):
        """ names() is a django-taggit method, returning a ValuesListQuerySet 
        (basically just an iterable) containing the name of each tag as a string
        """
        return self.tags.names()

    def __str__(self):
        return self.title

My serializers.py:

class TagsField(serializers.Field):
    """ custom field to serialize/deserialize TaggableManager instances.
    """
    def to_representation(self, value):
        """ in drf this method is called to convert a custom datatype into a primitive,
        serializable datatype.

        In this context, value is a plain django queryset containing a list of strings.
        This queryset is obtained thanks to get_tags() method on the Task model.

        Drf is able to serialize a queryset, hence we simply return it without doing nothing.
        """
        return value

    def to_internal_value(self, data):
        """ this method is called to restore a primitive datatype into its internal
        python representation.

        This method should raise a serializers.ValidationError if the data is invalid.
        """
        return data

class TaskSerializer(serializers.ModelSerializer):

    # tags field in Task model is implemented via TaggableManager class from django-taggit.
    # By default, drf is not able to serialize TaggableManager to json.
    # get_tags() is a method of the Task model class, which returns a Queryset containing
    # the list of tags as strings. This Queryset can be serialized without issues.
    tags = TagsField(source="get_tags")

    class Meta:
        model = Task
        fields = [
                "name",
                ...,
                "tags",
            ]

Whenever I try to create a new instance of my Task model via POST api, I get the following error:

TypeError at /taskdrop/v1/task/
Got a `TypeError` when calling `Task.objects.create()`. This may be because you have a writable field on the serializer class that is not a valid argument to `Task.objects.create()`. You may need to make the field read-only, or override the TaskSerializer.create() method to handle this correctly.
Original exception was:
 Traceback (most recent call last):
  File "/home/daniele/prj/ea/TaskDrop/venv/lib/python3.8/site-packages/rest_framework/serializers.py", line 948, in create
    instance = ModelClass._default_manager.create(**validated_data)
  File "/home/daniele/prj/ea/TaskDrop/venv/lib/python3.8/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/daniele/prj/ea/TaskDrop/venv/lib/python3.8/site-packages/django/db/models/query.py", line 420, in create
    obj = self.model(**kwargs)
  File "/home/daniele/prj/ea/TaskDrop/venv/lib/python3.8/site-packages/django/db/models/base.py", line 501, in __init__
    raise TypeError("%s() got an unexpected keyword argument '%s'" % (cls.__name__, kwarg))
TypeError: Task() got an unexpected keyword argument 'get_tags'

I'm kinda stuck right now...the field is definitely not read-only and regarding overriding the TaskSerializer.create() method, I don't know precisely how to do that.

Plus, I'm a bit confused about TagsField(serializers.Field) vs .create() method override. From my understanding, if I create a custom serializers field, there should be no additional need to override .create().

Finally, I tried to use django-taggit-serializer without success: the model gets created but the passed tags are just missing.

How can I fix this? Thanks.

Answer 1

Ok, I managed to make it work.

Leaving here the solution for others:

The reson I was getting TypeError: Task() got an unexpected keyword argument 'get_tags' is because drf was trying to use the return value of to_internal_value() to fill in the 'get_tags' field of my model.

Now, 'get_tags' is just method name of my model Task class, not a real field, hence the error. Drf learned about 'get_tags' as a field name when I used tags = TagsField(source="get_tags") in my serializer.

I worked around this issue overriding the create() method of my serializer, in this way:

class TaskSerializer(serializers.ModelSerializer):

    # tags field in Task model is implemented via TaggableManager class from django-taggit.
    # By default, drf is not able to serialize TaggableManager to json.
    # get_tags() is a method of the Task model class, which returns a Queryset containing
    # the list of tags as strings. This Queryset can be serialized without issues.
    tags = TagsField(source="get_tags")
    # variables = VariableSerializer()

    def create(self, validated_data):
        # using "source=get_tags" drf "thinks" get_tags is a real field name, so the
        # return value of to_internal_value() is used a the value of a key called "get_tags" inside validated_data dict. We need to remove it and handle the tags manually.
        tags = validated_data.pop("get_tags")
        task = Task.objects.create(**validated_data)
        task.tags.add(*tags)

        return task

    class Meta:
        model = Task
        # we exclude all those fields we simply receive from Socialminer
        # whenever we get a task or its status
        fields = [
                "name",
                 ...
                "tags",
            ]

Answer 2

I think you may need some sort of tag serializer setup.

So in your TaskSerializer I would have: tags = TagSerializer(many=True, read_only=False)

from serializers import (
    TagListSerializerField,
    TagSerializer
)

class TaskSerializer(TagSerializer, serializers.ModelSerializer):

    # tags field in Task model is implemented via TaggableManager class from django-taggit.
    # By default, drf is not able to serialize TaggableManager to json.
    # get_tags() is a method of the Task model class, which returns a Queryset containing
    # the list of tags as strings. This Queryset can be serialized without issues.
    tags = TagListSerializerField()

    class Meta:
        model = Task
        fields = [
                "name",
                ...,
                "tags",
            ]

I implemented this before many years ago, the TagList, TagListSerializerField and TagSerializer you want is this:

import six
import json

from django.utils.translation import ugettext_lazy as _

from rest_framework import serializer

class TagList(list):
    def __init__(self, *args, **kwargs):
        pretty_print = kwargs.pop("pretty_print", True)
        list.__init__(self, *args, **kwargs)
        self.pretty_print = pretty_print

    def __add__(self, rhs):
        return TagList(list.__add__(self, rhs))

    def __getitem__(self, item):
        result = list.__getitem__(self, item)
        try:
            return TagList(result)
        except TypeError:
            return result

    def __str__(self):
        if self.pretty_print:
            return json.dumps(
                self, sort_keys=True, indent=4, separators=(',', ': '))
        else:
            return json.dumps(self)


class TagListSerializerField(serializers.Field):
    child = serializers.CharField()
    default_error_messages = {
        'not_a_list': _(
            'Expected a list of items but got type "{input_type}".'),
        'invalid_json': _('Invalid json list. A tag list submitted in string'
                          ' form must be valid json.'),
        'not_a_str': _('All list items must be of string type.')
    }
    order_by = None

    def __init__(self, **kwargs):
        pretty_print = kwargs.pop("pretty_print", True)

        style = kwargs.pop("style", {})
        kwargs["style"] = {'base_template': 'textarea.html'}
        kwargs["style"].update(style)

        super(TagListSerializerField, self).__init__(**kwargs)

        self.pretty_print = pretty_print

    def to_internal_value(self, value):
        if isinstance(value, six.string_types):
            value = value.split(',')

        if not isinstance(value, list):
            self.fail('not_a_list', input_type=type(value).__name__)

        for s in value:
            if not isinstance(s, six.string_types):
                self.fail('not_a_str')

            self.child.run_validation(s)
        return value

    def to_representation(self, value):
        if not isinstance(value, TagList):
            if not isinstance(value, list):
                if self.order_by:
                    tags = value.all().order_by(*self.order_by)
                else:
                    tags = value.all()
                value = [tag.name for tag in tags]
            value = TagList(value, pretty_print=self.pretty_print)

        return value


class TagSerializer(serializers.Serializer):
    def create(self, validated_data):
        to_be_tagged, validated_data = self._pop_tags(validated_data)

        tag_object = super(TaggitSerializer, self).create(validated_data)

        return self._save_tags(tag_object, to_be_tagged)

    def update(self, instance, validated_data):
        to_be_tagged, validated_data = self._pop_tags(validated_data)

        tag_object = super(TaggitSerializer, self).update(
            instance, validated_data)

        return self._save_tags(tag_object, to_be_tagged)

    def _save_tags(self, tag_object, tags):
        for key in tags.keys():
            tag_values = tags.get(key)
            getattr(tag_object, key).set(*tag_values)

        return tag_object

    def _pop_tags(self, validated_data):
        to_be_tagged = {}

        for key in self.fields.keys():
            field = self.fields[key]
            if isinstance(field, TagListSerializerField):
                if key in validated_data:
                    to_be_tagged[key] = validated_data.pop(key)

        return (to_be_tagged, validated_data)