Using Google App Engine Queues service with Django

Question

I am trying to use Google App Engine queues API, I am having problems on testing this. It seems that in some part of the process the CSRF it's not working.

as I understand the api executes the task calling the url and making and http request in background.

The complete url is the API is calling is → http://localhost.localdomain:8000/admin/cooking/recipe/36/chefworker/

When it raises this exception:

Traceback (most recent call last):
  File "/home/mariocesar/Proyectos/Cooking/cooking/django/core/handlers/base.py", line 100, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/home/mariocesar/Proyectos/Cooking/cooking/django/views/decorators/csrf.py", line 24, in wrapped_view
    resp.csrf_exempt = True
AttributeError: 'NoneType' object has no attribute 'csrf_exempt'

So, the csrf middleware, the cookie, some data or the response itself is missing from the request that the GAE api makes to execute the task in the background.

How to solve this without disabling CSRF on Django? however, it's posible with djangoappengine at all?

Down are the models.py and admin.py files I am using.

models.py

from django.db import models

class Recipe(models.Model):
    name = models.CharField(max_length=140)
    description = models.TextField()
    cooking_time = models.PositiveIntegerField()
    status = models.CharField(max_length=40)

    def __unicode__(self):
        return self.name

    def cookthis(self):
        import time
        self.status = 'The chef is cooking this recipe'
        self.save()
        time.sleep(obj.cooking_time)
        self.status = 'It\'s done ! the recipe is ready to serve'
        self.save()

admin.py

import logging

from django.contrib import admin, messages
from django.http import HttpResponse
from django.utils.functional import update_wrapper
from django.contrib.admin.util import unquote
from django.shortcuts import get_object_or_404, render_to_response
from django import template
from django.core.urlresolvers import reverse
from google.appengine.api import taskqueue
from google.appengine.api.taskqueue import TaskAlreadyExistsError

from cooking.models import Recipe
from django.views.decorators.csrf import csrf_exempt

class AdminRecipe(admin.ModelAdmin):
    def get_urls(self):
        from django.conf.urls.defaults import patterns, url

        def wrap(view):
            def wrapper(*args, **kwargs):
                return self.admin_site.admin_view(view)(*args, **kwargs)
            return update_wrapper(wrapper, view)

        info = self.model._meta.app_label, self.model._meta.module_name

        urlpatterns = super(AdminRecipe, self).get_urls()
        myurls = patterns('',
            url(r'^(.+)/cook/$',
                wrap(self.cook_view),
                name='%s_%s_chefworker' % info),
            url(r'^(.+)/chefworker/$',
                wrap(self.chefworker_worker),
                name='%s_%s_chefworker' % info),
        )
        return myurls + urlpatterns

    def cook_view(self, request, object_id, extra_context=None):
        obj = get_object_or_404(Recipe, pk=unquote(object_id))
        if request.POST:
            try:
                taskqueue.add(
                    name="recipie-%s" % obj.id,
                    url=reverse('admin:cooking_recipe_chefworker', args=(obj.id,))
                )
                messages.add_message(request, messages.INFO, 'Chef is cooking the recipe.')
            except TaskAlreadyExistsError:
                messages.add_message(request, messages.ERROR, 'chef is already cooking that recipe.')

        context_instance = template.RequestContext(request, current_app=self.admin_site.name)
        return render_to_response("admin/cooking/recipe/cook_view.html", {'object': obj}, context_instance=context_instance)

    #TODO: Add csrf token on form
    @csrf_exempt
    def chefworker_worker(self, request, object_id, extra_context=None):
        import time

        if request.POST:
            obj = get_object_or_404(Recipe, pk=unquote(object_id))
            obj.cookthis()

        return HttpResponse('done')

admin.site.register(Recipe, AdminRecipe)

IMPORTANT NOTE: Was hard to debug this error, cause the dev_appserver logger was just raising 403 errors, no other info; so, I have to patch the file google/appengine/api/taskqueue/taskqueue_stub.py line 574 and add "logging.info('response --- \n%s' % result)" to get the output.

Answer 1

If you have the CsrfViewMiddleware enabled, Django will require a csrf_token in all POSTs to your views.

Django provides a decorator, @csrf_exempt, that you should place on your task queue views. This turns off the middleware just for those views.

Alternatively, you can avoid using CsrfViewMiddleware altogether and instead use the @csrf_protect decorator where you need it. I don't recommend doing this -- it's probably safer to protect everywhere and carve out a small number of exemptions for your task queue views.

(One last note: both answers above -- that something is wrong with your view, or that you should just use GET for the task queue -- strike me wrong. There's nothing wrong with your view, and POST is the right verb to use for task queue tasks.)

Answer 2

Looking at the source of csrf.py, it looks like this would only occur if your view function is returning None (or not explicitly returning, in which case Python would return None implicitly). Looking at your code, I don't see how that could occur, though - are you sure this is your exact deployed code?

Also, you probably don't want to use get_object_or_404 inside a task queue task - if it can't find the object, it'll throw a 404, which will cause the task to error and retry indefinitely.

You also shouldn't need CSRF protection (per your TODO); instead, make sure the task queue URL is marked admin-only, and it will only ever be called by the task queue service.

Answer 3

I'm not an expert, but you may try using GET instead of POST. See http://groups.google.com/group/django-non-relational/browse_thread/thread/e6baed5291aed957/d6c42150c8e246e1?lnk=gst&q=queue#d6c42150c8e246e1 (the last entry)