CODEWITHSUNDEEP
google-app-enginegogoogle-cloud-datastore
fisker
fisker

Reputation: 1039

Datastore KeysOnly pricing & implementation in Go

I know KeysOnly queries are cheaper, but I'm wondering if I'm implementing it correctly.

For instance, if I make a GetAll func for KeysOnly (like the one below), should I still initialize the users struct ([]User)?

func GetAllUserKeysByFilter(ctx context.Context, filters ...UserFilter) ([]*datastore.Key, error) {
    var users []User
    query := datastore.NewQuery("User").KeysOnly()
    for _, filter := range filters {
        if filter.Age != 0 {
            query = query.Filter("Age =", filter.Age)
        }
        if filter.Limit != 0 {
            query = query.Limit(filter.Limit)
        }
    }
    keys, err := DatastoreClient().GetAll(ctx, query, &users)
    if err != nil {
        return nil, err
    }
    return keys, nil
}

Moreover, what if I only want to get key of a direct call (where I already know the namekey - in this case, namekey == username) for the purpose of checking if this key exists or not, then what's the right way to do it?

Right now I do like this:

func GetUserByNameKey(ctx context.Context, key *datastore.Key) (User, error) {
    var user User
    err := DatastoreClient().Get(ctx, key, &user)
    if err != nil {
        return user, err
    }
    return user, nil
}

But can it be converted somehow to become cheaper since I only care about finding out if the User key exists or not?

Maybe it would be cheaper to use the GetAllUserKeysByFilter with Limit == 1?

Upvotes: 1

Views: 318

Answers (1)

icza
icza

Reputation: 418575

Query / Get operation cost

The cost for a Get operation (get by key) is 1 read op (and no small ops). Whether there is an actual entity for the key or not does not matter in terms of cost: both will cost you 1 read op. The "hard" part is finding the entity for the key, or finding out that there is no such entity, returning the result is the "easy" part.

The cost for keys-only queries is 1 read op + 1 small op for each result (for each result key).

For completeness, the cost for a "normal" query (meaning not keys-only) is 1 read op + 1 read op for each returned entity.

Example: A normal query returning 10 entities is 11 read ops. A keys-only query returning 10 keys is 1 read op + 10 small ops. A Get call is 1 read op (regardless of whether the entity is found and returned).

So calling GetAllUserKeysByFilter() with limit = 1 would theoretically cost more: it costs 1 read operation and 1 small operation. But since small operations are free (cost no real money), they are basically equal. But know that a Get() operation (by key) is likely to be faster unless your entity is large. If your entity is large, use a keys-only query with limit = 1. If your entity is small, Get() will most likely be faster (measure if it does matter to you).

Some information about query costs can be found on this page: Datastore Queries

Keys-only query example

Your implementation of the keys-only query is unnecessarily complex. When you're executing a keys-only query, the dst destination parameter of Query.GetAll() is omitted (not used), only the return value is useful which is exactly that: the slice of result keys:

If q is a “keys-only” query, GetAll ignores dst and only returns the keys.

So basically your solution also works, but the users slice will not be used. Your GetAllUserKeysByFilter() function should look like this:

func GetAllUserKeysByFilter(ctx context.Context, filters ...UserFilter)
        ([]*datastore.Key, error) {

    query := datastore.NewQuery("User").KeysOnly()
    for _, filter := range filters {
        if filter.Age != 0 {
            query = query.Filter("Age =", filter.Age)
        }
        if filter.Limit != 0 {
            query = query.Limit(filter.Limit)
        }
    }
    keys, err := DatastoreClient().GetAll(ctx, query, nil)
    if err != nil {
        return nil, err
    }
    return keys, nil
}

P.S. If you want to reuse GetAllUserKeysByFilter() for both keys-only and normal queries, you may continue to do so, but then you should also return the users slice.

Upvotes: 2

Related Questions