Django uploading and importing a file

Question

I have a Django app that is basically a web front end on a database.

Every now and then, users upload files containing perhaps 1000s of records. These records will need to be parsed out of the file, processed, and used to create new records or update existing records in the database. I'm just wondering what is the better approach for processing the uploaded file:

• in the view (while the user waits - I guess this could be a up to 5 minues) ?
• save the uploaded file and have some background cron job call a custom admin command to process it? This seems the most sensible to me.
• or perhaps another method I haven't thought of?

Answer 1

Celery seems to be pretty hot these days too, you should definitely look into this:

https://github.com/ask/django-celery

http://celeryproject.org/

Send an email when done, or have the front end poll for results every X seconds after submission. "Are we there yet?" "Are we there yet?"

---

I'd like to know too a simple, safe way to start a thread that writes to the db.