Django UnicodeDecodeError

Question

I'm pretty new to Django and keep getting this error and can't for the life of me, figure out the solution. I think I've included all the relevant possible code sections and any help would really be appreciated! The error occurs when I'm trying to print out all the students in a schoolclass. I think the error is caused by something related to the line render(request, 'schoolclass/students.html', context). Here are the relevant sections of my app, along with the error message.

schoolclass.views.py

def detail(request, schoolclass_id):
  try:
    student_list = Student.objects.filter(schoolclass_id = schoolclass_id).order_by('lastname')
    schoolclass = SchoolClass.objects.get(id = schoolclass_id)
    context = {'student_list': student_list, 'schoolclass': schoolclass}
  except Student.DoesNotExist:
    raise Http404
  return render(request, 'schoolclass/students.html', context)

schoolclass.urls.py

urlpatterns = patterns('',
                       url(r'^$', views.index, name='index'),
                       url(r'^(?P<schoolclass_id>\d+)/$', views.detail, name='detail'),
                       )

students.html

{% block content %}

<h1>{{ schoolclass.yearlevel }} {{ schoolclass.subject }} {{ schoolclass.description }}</h1>

{% if error_message %}<p><strong>{{ error_message }}</strong></p>{% endif %}

<table>
    <tr>
            <th>Last Name</th>
            <th>First Name</th>
    </tr>
    {% for student in student_list %}
    <tr>
        <td>{{ student.lastname }}</td>
        <td>{{ student.firstname }}</td>
    </tr>
    {% endfor %}
    <tr>
        <td>{{ student.lastname }}</td>
        <td>{{ student.firstname }}</td>
    </tr>
</table>

{% endblock %}

Error Message

Request Method: GET
Request URL: http://127.0.0.1:8000/schoolclass/1/


Traceback:
File "C:\Python27\lib\site-packages\django\core\handlers\base.py" in get_response
  115.                         response = callback(request, *callback_args,    **callback_kwargs)
File "c:\Code\markbook\schoolclass\views.py" in detail
  22.   return render(request, 'schoolclass/students.html', context)
File "C:\Python27\lib\site-packages\django\shortcuts\__init__.py" in render
  53.     return HttpResponse(loader.render_to_string(*args, **kwargs),
File "C:\Python27\lib\site-packages\django\template\loader.py" in render_to_string
  170.         t = get_template(template_name)
File "C:\Python27\lib\site-packages\django\template\loader.py" in get_template
  146.     template, origin = find_template(template_name)
File "C:\Python27\lib\site-packages\django\template\loader.py" in find_template
  135.             source, display_name = loader(name, dirs)
File "C:\Python27\lib\site-packages\django\template\loader.py" in __call__
  43.         return self.load_template(template_name, template_dirs)
File "C:\Python27\lib\site-packages\django\template\loader.py" in load_template
  46.         source, display_name = self.load_template_source(template_name, template_dirs)
File "C:\Python27\lib\site-packages\django\template\loaders\filesystem.py" in load_template_source
  38.                     return (fp.read().decode(settings.FILE_CHARSET), filepath)
File "C:\Python27\lib\encodings\utf_8.py" in decode
  16.     return codecs.utf_8_decode(input, errors, True)

Exception Type: UnicodeDecodeError at /schoolclass/1/
Exception Value: 'utf8' codec can't decode byte 0x85 in position 702: invalid start byte

models

class SchoolClass(models.Model):
  user = models.ForeignKey(User)
  subject = models.CharField("Subject", max_length=100, choices = SUBJECT_CHOICES, default='Select One')
  yearlevel = models.CharField("Year Level", max_length=100, choices = YEARLEVEL_CHOICES, default='Select One')
  description = models.CharField("Unique identifier", max_length=100, default='Maybe 2013 or school classcode')


class Student(models.Model):
  schoolclass = models.ForeignKey(SchoolClass) 
  firstname = models.CharField(max_length=50)
  lastname = models.CharField(max_length=50)

Answer 1

I changed encoding on my main base html file to UTF16 which resulted to the error in template rendering that is the and this changed the file encoding settings. Yes, I ran into the error then landed here antonis helped me solve the mess, by forcing the encoding to utf 8 just by using my editor (pycharm) on the bottom part there is encoding settings, just convert the current encoding to utf 8 and the error will be gone

Answer 2

I should also point out that you might also get this error if there is any non-ascii character in your file path. Say something like:

D:\säga\något\my_project\templates

Renaming the file to include only ascii characters should solve the problem.

Answer 3

This part of the traceback:

File "C:\Python27\lib\site-packages\django\template\loaders\filesystem.py" in load_template_source
  38.                     return (fp.read().decode(settings.FILE_CHARSET), filepath)

indicates that the error occurred while loading the template from the disk, not while rendering the template.

In addition, the error message:

Exception Value: 'utf8' codec can't decode byte 0x85 in position 702

indicates that the problem is in position 702 of the file. However, your pasted students.html is only about 560 bytes. Therefore, either you haven't pasted the entire file, or it's actually reading a different file than the one you think.

Answer 4

I think you have a problem with template file encoding. Try to open it without any data (empty student_list, some dummy schoolclass). If it will be still throwing an error, the problem is a template file itself so you just need to save it with you editor forcing utf-8.

Otherwise, if it will work fine with an empty context, you need to look for badly-encoded entries in your db. For this you can write a loop and check student_list elements one-by-one.