CODEWITHSUNDEEP
pythondjangopython-3.x
Kye
Kye

Reputation: 4495

Django FileField not cooperating with Python 3's csv module

I have this form:

class CSVImportForm(AuthorshipMixin, FormMixin, forms.Form):

    data = forms.FileField()

    def import_csv(self):
        school_csv = csv.DictReader(self.cleaned_data['data'])
        for row in school_csv:
            print(row)

The intended purpose of import_csv() is to import the CSV to the application's database, but it has been altered for brevity's sake.

An exception occurs when I attempt to iterate over school_csv, which I guess is when DictReader first attempts to read the file:

Traceback:
File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/base.py" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.5/dist-packages/django/views/generic/base.py" in view
  71.             return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/django/views/generic/base.py" in dispatch
  89.         return handler(request, *args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/django/views/generic/edit.py" in post
  215.             return self.form_valid(form)
File "/opt/project/project/import/views.py" in form_valid
  16.         form.import_csv()
File "/opt/project/project/import/forms.py" in import_csv
  67.         for row in school_csv:
File "/usr/lib/python3.5/csv.py" in __next__
  109.             self.fieldnames
File "/usr/lib/python3.5/csv.py" in fieldnames
  96.                 self._fieldnames = next(self.reader)

Exception Type: Error at /import/
Exception Value: iterator should return strings, not bytes (did you open the file in text mode?)

I don't believe that I was presented with the opportunity to ever choose the mode to open the file with. How am I able to open in text mode so the csv module will handle it?

I've noticed that InMemoryUploadedFile has a file attribute which seemed useful, but it's a io.BytesIO object. I imagine I could do some magic involving manually specifying the encoding.etc but I imagined that this is the sort of thing that Django would make easier.

What is the best way to approach this?

Upvotes: 2

Views: 1234

Answers (2)

cab404
cab404

Reputation: 161

There's a also a variant with streaming:

reader = csv.DictReader(io.TextIOWrapper(request.FILE["csv"]))

It should work better with huge stuff.

Upvotes: 2

Art
Art

Reputation: 2335

Try using

as_string = self.cleaned_data['data'].read().decode("utf-8")
school_csv = csv.DictReader(as_string)

if it expects str or

from io import StringIO
school_csv = csv.DictReader(StringIO(as_string))

if it wants StringIO.

Upvotes: 7

Related Questions