Django deployment - local and remote

Question

I have a django project, using postgres, for which 85%+ of uses occur inside a LAN. The other 15% are from the internet. Occasionally, WAN use cases jut up to 90% for a given hour or couple hours.

The project is just a general-purpose community application that serves a small town in mid-state NY (New Paltz).

Delivery of large files or streaming media over our local 54Mbps is pretty great; we'd like to do something like public access TV and radio pretty soon.

Performance is not the only concern; we also want to be able to use the app in the complete absence of an Internet connection. This is mostly an academic exercise, but not completely: we loose our connection a few times a year during storms and we'd love to still be up.

Here's the scenario for deployment (that I think) I desire: I'd like to run the django project both locally and on rackspace. I'd like static content to be hosted in both places - obviously local users will get the local version, others will get the rackspace hosted version.

I'd like the rackspace instance to allow database writes, but to transmit them to my local DB as soon as is convenient, keeping the local DB as the authoritative one.

Is this feasible? Is it good practice? Is it well documented? Is there a better easy way?

Answer 1

I think the reason you are not getting the answer you want is because the deployment you describe really does sound like adding a significant level of complexity for a marginal benefit (in performance). Also you haven't mentioned what type of application it is, or what flavor of database server you run.

Having said that, here's some general advice.

If the only reason you don't want to host the app with an external provider is speed (for local users), then consider a few ways to make your app run faster, no matter where it's deployed.

I've seen massive inmprovements in the performance of Django applications through a few simple steps:

1. Analyse database queries using django-debug-toolbar, ensure that you have created indexes on all columns that appear in WHERE clauses (unless those column are particularly write heavy)

2. Make sure your servers memory isn't over allocated (free -m on linux, look at the second row).

3. Make sure you are 'long-caching' static content (and ideally serving from a CDN).

4. If your cloud hosting provider offers database servers, try them out. They're probably hundreds of times faster than what you could achieve on your own VPS.

5. Again, with django-debug-toolbar, see which SQL queries take a long time to run. Consider, are these queries cachable? If so, use django's low-level caching API to prevent unnecessarily rerunning expensive db queries.

Also, if possible - pick a cloud hosting provide that is geographically close to you. The difference between 300ms latency and 8ms latency is huge.

After doing all this, if you still want a local mirror of a remote database, it can be done.

Firstly, look into your databases replication features; be very cautious about adding unnecessary complexity. One-way DB replication to a readonly database is pretty much an order of magnitude simpler than anything else.

Django allows you to configure database routers per-model, and per read / write. However be very cautious about going down the path of reading from one db and writing to another; no database replication scheme can guarantee a low enough latency as to not cause data inconsistencies.

However, you might find that you can get a pragmatic outcome by running your contrib.auth app (users, sessions) on the read/write database, but other, more 'reporting' type apps on the read only mirror.

I hope that gives you some ideas.

EDIT: Since you keep adding to / changing the question. I'm going to clarify a few things for you:

Is this feasible?

Yes. Given enough time and money, anything is possible.

Is it good practice?

Good practice generally can be judged by a number of thought tests. One such test is, "is this the simplest thing that can meet the requirement, without oversimplifying the requirement?".

I'd say no, this isn't good practice, if only because it probably isn't the simplest thing that can meet your requirement. Look for people in similar situations to yourselves, contact them, and ask them how they do it.

Is it well documented?

Uncommon deployments are rarely well documented, this is definitely an uncommon proposal.

Is there a better easy way?

Yes, absolutely. Complexity is your enemy. As a rule of thumb every time you add a single additional level of complexity, you double the testing required to safely make changes to your application. Yes you can automate these tests, but writing tests is also a lot of work. Better to avoid the complexity in the first place.

Closing Thoughts

Your question has evolved from, "can I run two geographically distributed instances of a web site, each with it's own database - but only one authoritative" to something much more complicated.

The fact that it's a complicated question says to me that you need to simplify your thinking as to what is the actual requirement. Can you think of a test that your system will pass, that it doesn't today, if you achieve the benefit that you must deliver?

Answer 2

Feasible? Of course.

Good Practice? If you can afford the cost/speed of a rackspace-only deployment, I'd just run it in the cloud.

Well documented? The various pieces (replication, automated deployment, etc.) are. You'll have to bring them together to get your specific use case.

Is there a better way? Depends... The only major pain point I see is keeping the database in sync. A potential way around that would to have the rackspace servers point at your LAN DB through some firewall/vpning. Then you just have web servers in the cloud w/a deployment script to make all the necessary updates. Problem is if your internal network goes down, you're kind of screwed. If this is just for internal company use, that downtime may be acceptable. It's hard to say without knowing your specific use case.