CODEWITHSUNDEEP
python-2.7google-app-enginegoogle-cloud-datastore
beingjainparas
beingjainparas

Reputation: 9

how to calculate the sum of a column of google datastore

i am writing a application in webbapp2 framework

my database is

class SurveyFaculty(db.Model):
name = db.StringProperty()
clas = db.StringProperty()
roll_no = db.IntegerProperty()
semester = db.IntegerProperty()
q11 = db.IntegerProperty()
q12 = db.IntegerProperty()
q13 = db.IntegerProperty()
q14 = db.IntegerProperty()
q15 = db.IntegerProperty()
q21 = db.IntegerProperty()
q22 = db.IntegerProperty()
q23 = db.IntegerProperty()
q24 = db.IntegerProperty()
q25 = db.IntegerProperty()
q31 = db.IntegerProperty()
q32 = db.IntegerProperty()
q33 = db.IntegerProperty()
q34 = db.IntegerProperty()
q35 = db.IntegerProperty()
q41 = db.IntegerProperty()
q42 = db.IntegerProperty()
q43 = db.IntegerProperty()
q44 = db.IntegerProperty()
q45 = db.IntegerProperty()
q51 = db.IntegerProperty()
q52 = db.IntegerProperty()
q53 = db.IntegerProperty()
q54 = db.IntegerProperty()
q55 = db.IntegerProperty()
q61 = db.IntegerProperty()
q62 = db.IntegerProperty()
q63 = db.IntegerProperty()
q64 = db.IntegerProperty()
q65 = db.IntegerProperty()
q71 = db.IntegerProperty()
q72 = db.IntegerProperty()
q73 = db.IntegerProperty()
q74 = db.IntegerProperty()
q75 = db.IntegerProperty()
q81 = db.IntegerProperty()
q82 = db.IntegerProperty()
q83 = db.IntegerProperty()
q84 = db.IntegerProperty()
q85 = db.IntegerProperty()
q91 = db.IntegerProperty()
q92 = db.IntegerProperty()
q93 = db.IntegerProperty()
q94 = db.IntegerProperty()
q95 = db.IntegerProperty()
q101 = db.IntegerProperty()
q102 = db.IntegerProperty()
q103 = db.IntegerProperty()
q104 = db.IntegerProperty()
q105 = db.IntegerProperty()
q111 = db.IntegerProperty()
q112= db.IntegerProperty()
q113= db.IntegerProperty()
q114= db.IntegerProperty()
q115= db.IntegerProperty()
q121= db.IntegerProperty()
q122= db.IntegerProperty()
q123= db.IntegerProperty()
q124 = db.IntegerProperty()
q125 = db.IntegerProperty()
created = db.DateTimeProperty(auto_now_add = True)

i want to know what query should i write if i want too get the addition (sum) of a particular column???

how can i calculate sum of each column of my database???

Upvotes: 0

Views: 1389

Answers (4)

Prateek Jain
Prateek Jain

Reputation: 1552

SUM aggregation in datastore is generally available now.

There are client libraries available in multiple languages which support this particular feature.

With Aggregation queries, users can avoid performing the client side aggregations which puts an additional burden of increased egress cost. Also no need of using alternatives like cloud-functions to update the aggregate values on the backend side which has a cost limitation of their own.

Upvotes: 0

Patrick Costello
Patrick Costello

Reputation: 3626

As an alternative to using a map reduce, you can pre-calculate the sum for properties you care about. In particular, you can use a sharded counter.

This will let you efficiently track the sum of your properties without having to run any jobs to do the calculation (except maybe an initial map reduce to populate the sums for existing data). The sharded counter has the benefit that it can scale to prevent contention if you are writing a lot of entities that need to update the counter.

Upvotes: 1

Daniel Roseman
Daniel Roseman

Reputation: 599610

You can't. GAE datastore is not a relational database, you can't do sums.

The best bet is to iterate through and calculate the sum with the mapreduce framework.

Upvotes: 3

Patrice
Patrice

Reputation: 4692

As was said before, you simply can't.

How the GAE datastore is built, don't even think about ANY server-side computation, it simply won't do it. The speed and scalability of the datastore comes from the fact it only does scans on pre-sorted tables (you lose a bit of speed on writes, but gain an incredible amount in reads).

So you have the choice of either keeping a separate entity for your total and update that entity whenever you push new data in (have "id = 1" be your total and when you put a new, you get id 1 and do += on each property before putting id 1 in the datastore again), or use mapreduce, as was mentioned by Daniel

Upvotes: 2

Related Questions