Google App Engine Datastore Entity Design for Minimizing Cost and Enhancing Performance

Question

I have been writing an App Engine Application and lately started to have some concerns about the cost of App Engine when its data scale up.

My application has employee data. The Employee entity has so many properties such as first name, last name, birthday, address, phone, mobile phone, reference persons, education backgrounds, training history, income data and so on. Total properties could reach up to 100 properties in the future.

At first, I thought that I should not pack all the properties in one entity which was the Employee entity because I would have to get all the properties in the entity every time when I want to put or update just a couple properties in the entity, so I broke up the entity into many entities in one-to-one relationship fashion and made the User Interface into tabs, contact information tab, education background tab, income tab and so on. When a user clicks on a tab, the data when then be loaded from that particular entity and the data on previous closed tab will be saved automatically. I divided a large entity into many small one-to-one relationship entities for performance and cost efficiency. I did this for the paging efficiency as well by getting just small entities with minimum property numbers for employee listing page.

After reading App Engine pricing model, it charges costs based on the times of writing and reading each entity which means the way I designed my entity will increase the number of writing and reading times. I thought that my design was cost effective and had a better performance. I use less CPU and bandwidth for each time I get an entity since I do not need all of the properties to be displayed on the same page at a time.

Or should I compact the entities into one or a couple entities to improve my application performance and reduce the costs. And for the paging functionality I could create a specific-purpose entity for handling the paging manually.

Answer 1

Your approach is correct if your Employee entity has many indexed properties. Every time you update an entity, you incur write costs for each indexed property. So splitting an entity into smaller pieces will significantly reduce the write costs over the life of an entity. In other words, the initial cost of writing all this data for the first time might be slightly higher, but each subsequent update will be significantly cheaper.

Similar logic applies to reads. Multiple entities will be slightly more expensive only if your users will click on every tab for every employee, which is highly unlikely. Most of the time, they will go to a specific tab as they need to read data or make changes.

NB: I must also observe that unless you have hundreds of thousands of users, the dollar difference between these approaches is probably not worth your time.