Issue With Nested Comments Using MPTT in Django and Django Rest Framework API - Result In Detail Not Found

Question

I'm trying to create a nested comment system using MPTT but using Django Rest Framework to serialize MPTT tree. I got the nested comments to work - and these comments are added, edited, and deleted by calling Django Rest Framework API endpoints only - not using Django ORM DB calls at all. Unfortunately, there is a bug I couldn't figure out! Although the comments are added, edited, and deleted fine - but when a seventh or eighth comment is nested - suddenly the first-in comment or first-in nested comments would become [detail: Not found.] - meaning it will return an empty result or throw an unknown validation error somewhere which I couldn't figure out why. This results in when clicking on edit or delete the buggy comments becoming impossible - but the GET part is fine since these buggy comments do show up in the comment section (or should I say the list part returns fine). The image I'll attach will show that when I entered comment ggggg, the comment aaaa and bbbb will throw errors when trying to edit or delete them. If I delete comment gggg, comment hhhh will also be deleted (as CASCADE was enabled) - and suddenly comment aaaa and bbbb will work again for deletion and editing.

My comment model (models.py):

from django.db import models
from django.template.defaultfilters import truncatechars
from mptt.managers import TreeManager

from post.models import Post
from account.models import Account
from mptt.models import MPTTModel, TreeForeignKey


# Create your models here.

# With MPTT
class CommentManager(TreeManager):
    def viewable(self):
        queryset = self.get_queryset().filter(level=0)
        return queryset


class Comment(MPTTModel):
    parent = TreeForeignKey('self', on_delete=models.CASCADE, null=True, blank=True, related_name='comment_children')
    post = models.ForeignKey(Post, on_delete=models.CASCADE, related_name='comment_post')
    user = models.ForeignKey(Account, on_delete=models.CASCADE, related_name='comment_account')
    content = models.TextField(max_length=9000)
    created_date = models.DateTimeField(auto_now_add=True)
    updated_date = models.DateTimeField(auto_now=True)
    status = models.BooleanField(default=True)

    objects = CommentManager()

    def __str__(self):
        return f'Comment by {str(self.pk)}-{self.user.full_name.__self__}'

    @property
    def short_content(self):
        return truncatechars(self.content, 99)

    class MPTTMeta:
        # If changing the order - MPTT needs the programmer to go into console and do Comment.objects.rebuild()
        order_insertion_by = ['-created_date']

My serializers.py (Showing only comment serializer portion).

class RecursiveField(serializers.Serializer):
    def to_representation(self, value):
        serializer = self.parent.parent.__class__(value, context=self.context)
        return serializer.data

class CommentSerializer(serializers.ModelSerializer):
    post_slug = serializers.SerializerMethodField()
    user = serializers.StringRelatedField(read_only=True)
    user_name = serializers.SerializerMethodField()
    user_id = serializers.PrimaryKeyRelatedField(read_only=True)

    comment_children = RecursiveField(many=True)

    class Meta:
        model = Comment
        fields = '__all__'

    # noinspection PyMethodMayBeStatic
    # noinspection PyBroadException
    def get_post_slug(self, instance):
        try:
            slug = instance.post.slug
            return slug
        except Exception:
            pass

    # noinspection PyMethodMayBeStatic
    # noinspection PyBroadException
    def get_user_name(self, instance):
        try:
            full_name = f'{instance.user.first_name} {instance.user.last_name}'
            return full_name
        except Exception:
            pass

    # noinspection PyMethodMayBeStatic
    def validate_content(self, value):
        if len(value) < COM_MIN_LEN:
            raise serializers.ValidationError('The comment is too short.')
        elif len(value) > COM_MAX_LEN:
            raise serializers.ValidationError('The comment is too long.')
        else:
            return value

    def get_fields(self):
        fields = super(CommentSerializer, self).get_fields()
        fields['comment_children'] = CommentSerializer(many=True, required=False)
        return fields

The API views for comments would look like this:

class CommentAV(mixins.CreateModelMixin, generics.GenericAPIView):
    # This class only allows users to create comments but not list all comments.  List all comments would
    # be too taxing for the server if the website got tons of comments.
    queryset = Comment.objects.viewable().filter(status=True)
    serializer_class = CommentSerializer

    def post(self, request, *args, **kwargs):
        return self.create(request, *args, **kwargs)

    def perform_create(self, serializer):
        # Overriding perform_create.  Can create comment using the authenticated account.
        # Cannot pretend to be someone else to create comment on his or her behalf.
        commenter = self.request.user
        now = timezone.now()
        before_now = now - timezone.timedelta(seconds=COM_WAIT_TIME)
        # Make sure user can only create comment again after waiting for wait_time.
        this_user_comments = Comment.objects.filter(user=commenter, created_date__lt=now, created_date__gte=before_now)
        if this_user_comments:
            raise ValidationError(f'You have to wait for {COM_WAIT_TIME} seconds before you can post another comment.')
        elif Comment.objects.filter(user=commenter, level__gt=COMMENT_LEVEL_DEPTH):
            raise ValidationError(f'You cannot make another level-deep reply.')
        else:
            serializer.save(user=commenter)

    # By combining perform_create method to filter out only the owner of the comment can edit his or her own
    # comment -- and the permission_classes of IsAuthenticated -- allowing only authenticated user to create
    # comments.  When doing custome permission - such as redefinte BasePermission's has_object_permission,
    # it doesn't work with ListCreateAPIView - because has_object_permission is meant to be used on single instance
    # such as object detail.
    permission_classes = [IsAuthenticated]


class CommentAVAdmin(generics.ListCreateAPIView):
    queryset = Comment.objects.viewable()
    serializer_class = CommentSerializer

    permission_classes = [IsAdminUser]


class CommentDetailAV(generics.RetrieveUpdateDestroyAPIView):
    queryset = Comment.objects.viewable().filter(status=True)
    serializer_class = CommentSerializer

    permission_classes = [CustomAuthenticatedOrReadOnly]

    def destroy(self, request, *args, **kwargs):
        instance = self.get_object()
        if not instance.user.id == self.request.user.id:
            return Response({
                'Error': 'Comment isn\'t deleted! Please log into the owner account of this comment to delete this comment.'},
                status=status.HTTP_400_BAD_REQUEST)
        self.perform_destroy(instance)
        return Response({'Success': 'Comment deleted!'}, status=status.HTTP_204_NO_CONTENT)


class CommentDetailAVAdmin(generics.RetrieveUpdateDestroyAPIView):
    queryset = Comment.objects.viewable()
    serializer_class = CommentSerializer

    permission_classes = [IsAdminUser]


class CommentDetailChildrenAV(generics.RetrieveUpdateDestroyAPIView):
    queryset = Comment.objects.viewable().get_descendants().filter(status=True)
    serializer_class = CommentSerializer

    permission_classes = [CustomAuthenticatedOrReadOnly]

    def destroy(self, request, *args, **kwargs):
        instance = self.get_object()
        if not instance.user.id == self.request.user.id:
            return Response({
                'Error': 'Reply isn\'t deleted! Please log into the owner account of this reply to delete this reply.'},
                status=status.HTTP_400_BAD_REQUEST)
        self.perform_destroy(instance)
        return Response({'Success': 'Comment deleted!'}, status=status.HTTP_204_NO_CONTENT)

The API calls would look like this in blog_post app views:

add_comment = requests.post(BLOG_BASE_URL + f'api/post-list/comments/create-comments/',
                                                headers=headers,
                                                data=user_comment)
                                                
add_reply = requests.post(BLOG_BASE_URL + f'api/post-list/comments/create-comments/',
                                              headers=headers,
                                              data=user_reply)

requests.request('PUT', BLOG_BASE_URL + f'api/post-list/comments/{pk}/',
                                 headers=headers,
                                 data=user_comment)

response = requests.request('PUT', BLOG_BASE_URL + f'api/post-list/comments/children/{pk}/',
                                            headers=headers,
                                            data=user_comment)

response = requests.request("DELETE", BLOG_BASE_URL + f'api/post-list/comments/{pk}/', headers=headers)

These calls in the blog post app views would allow me to allow authenticated users to create, edit, and delete comments.

Does anyone know why my application got this bug? Any help would be appreciated! I read somewhere about getting a node refresh_from_db() - but how would I do that in the serialization? Also, Comment.objects.rebuild() doesn't help! I also noticed that when I stopped the development server and restarted it, the whole comment tree worked normally again - and I could now edit and delete the non-working comments earlier.

Update: I also opened up python shell (by doing Python manage.py shell) and tried this for the specific affected comment that when doing API call for edit or delete and got error of Not Found:

from comment.models import Comment
reply = Comment.objects.get(pk=113)
print(reply.content)

I did get the proper output of the comment's content. Then I also tried to get_ancestors(include_self=True) (using MPTT instance methods) - and I got proper output that when using include_self=True does show the affected comment's node in the output - but calling API endpoint results in Not Found (for GET) still.

I'm super confused now! Why? If I restart the development server by doing Ctrl-C and python manage.py runserver - and revisit the same affected API GET endpoint - this case is comment (child node) with 113 primary key(id) - the endpoint would show proper output and details as if nothing had gone wrong.

Update 2: Found an interesting Github post: https://github.com/django-mptt/django-mptt/issues/789 This sounds like what I'm experiencing but I'm not using Apache - and this is Django's default development server.

Answer 1

Okie, I figured it out!

I think when calling the same object in the Tree of MPTT for GET and PUT somehow spits out a weird bug that prevents me from editing the affected replies. So, my solution now is just creating an endpoint with API view below:

class CommentChildrenAV(mixins.CreateModelMixin, generics.GenericAPIView):
    # This class only allows users to create comments but not list all comments.  List all comments would
    # be too taxing for the server if the website got tons of comments.
    queryset = Comment.objects.viewable().get_descendants().filter(status=True)
    serializer_class = CommentSerializer

    def get(self, request, pk):
        replies = Comment.objects.viewable().get_descendants().filter(status=True, pk=pk)
        serializer = CommentSerializer(replies, many=True)
        return Response(serializer.data, status=status.HTTP_200_OK)

    def post(self, request, *args, **kwargs):
        return self.create(request, *args, **kwargs)

    def perform_create(self, serializer):
        # Overriding perform_create.  Can create comment using the authenticated account.
        # Cannot pretend to be someone else to create comment on his or her behalf.
        commenter = self.request.user
        now = timezone.now()
        before_now = now - timezone.timedelta(seconds=COM_WAIT_TIME)
        # Make sure user can only create comment again after waiting for wait_time.
        this_user_comments = Comment.objects.filter(user=commenter, created_date__lt=now, created_date__gte=before_now)
        if this_user_comments:
            raise ValidationError(f'You have to wait for {COM_WAIT_TIME} seconds before you can post another comment.')
        elif Comment.objects.filter(user=commenter, level__gt=COMMENT_LEVEL_DEPTH):
            raise ValidationError(f'You cannot make another level-deep reply.')
        else:
            serializer.save(user=commenter)

    # By combining perform_create method to filter out only the owner of the comment can edit his or her own
    # comment -- and the permission_classes of IsAuthenticated -- allowing only authenticated user to create
    # comments.  When doing custome permission - such as redefinte BasePermission's has_object_permission,
    # it doesn't work with ListCreateAPIView - because has_object_permission is meant to be used on single instance
    # such as object detail.
    permission_classes = [IsAuthenticated]

This API view would allow me to pass in the pk of the reply - get JSON response like so:

response = requests.request("GET", BLOG_BASE_URL + f'api/post-list/children/get-child/{pk}/', headers=headers)

Once I have the response in JSON - I could get the original reply content - input this reply content into reply-form's initial data like so:

edit_form = CommentForm(initial=original_comment_data)

Then I'm getting the POST's new content that the user wants to replace the original reply's content with - the gist is the solution I'm now going with is - if the user is authenticated and if the original's JSON content's user_id (meaning the original's commenter of the reply text) is the same as the request.user.id - then I just do:

if request.method == 'POST':
    # I can't use API endpoint here to edit reply because some weird bug won't allow me to do so.
    # Instead of calling the endpoint api for edit reply - I just update the database with
    # using ORM (Object Relational Manager) method.
    if request.user.is_authenticated:
        print(content[0]['user_id'], os.getcwd())
        if request.user.id == content[0]['user_id']:
            Comment.objects.filter(status=True, pk=pk).update(content=request.POST.get(strip_invalid_html('content')))
            post_slug = content[0]['post_slug']
            return redirect('single_post', post_slug)

This now solves my problem for real! I was just hoping that I don't have to cheat by going the route of ORM for editing a reply. I would prefer 100% API calls for all actions in this app. Sigh... but now my app is fully functioning in terms of having a comment system that is nested using MPTT package.