CODEWITHSUNDEEP
google-cloud-datastoreobjectify
Siva Reddy Vippala
Siva Reddy Vippala

Reputation: 211

GAE Datastore kind design with a requirement of too many entity fields

One of my datastore entity is growing with too many fields, hence it could be a future performance bottle neck.

As of now I see the entity consists of 100 fields, if I need to fetch 100 entities each having 100 fields, would definitely be a performance hit (considering underlying data serialization and de-serialization while fetching the data from data-store).

So is it a good idea to convert the entire entity to a blob and store it with a key value and later logically parse the data back into a required object format?

Any valuable suggestions please?

Upvotes: 0

Views: 53

Answers (2)

stickfigure
stickfigure

Reputation: 13556

Unless you have done some profiling and find that serialization is a real bottleneck, I wouldn't worry about how many fields you have. Object assembly and disassembly in Java is fast. In the unlikely event that you actually are hitting limits (say, thousands of entities with thousands of fields) you can write a custom Objectify translator that eliminates all the reflection overhead.

This sounds like premature optimization.

Upvotes: 2

bighonestjohn
bighonestjohn

Reputation: 328

I'm not so sure if converting the entity to a blob will increase the performance much, since you will still need to deserialise the blob into an entities later on in your application code.

If you never need all the fields of the object, then one method of increasing performance is to use projection queries. (See https://developers.google.com/appengine/docs/java/datastore/projectionqueries)

Projection queries basically allow you to return back only the properties you require. This works because it uses the information stored in the indexes, hence it never needs to deserialise the entity. This means that you must have an index defined for any projection query you use.

Upvotes: 1

Related Questions