CODEWITHSUNDEEP
google-app-enginegoogle-cloud-endpointsendpoints-proto-datastore
EsseTi
EsseTi

Reputation: 4271

Object with a long list of objects as a field w/endpoints-proto-datastore

I've to model a Club which has members of type User. Obviously there can be a huge number of members for a club.

i've done this

class User(EndpointsModel):
    username = ndb.StringProperty(required=True)


class Club(EndpointsModel):
    ....
    members_key = ndb.KeyProperty(kind="User", repeated=True)

    @EndpointsAliasProperty(repeated=True,property_type=User.ProtoModel())
    def members(self):
        return ndb.get_multi(self.members_key)

Done like this in the response i get the entire list of users, which in the case i've 5000 members can take a while.

Is there a possiblity to have this list paginated? maybe using ProtoCollection() instead of ProtoModel()? (i tried without success).

Or, how can i create an endpoint of the type /club/{id}/members that gives me back the list (paginated) of members?

Upvotes: 1

Views: 209

Answers (2)

Nick
Nick

Reputation: 3591

There are numerous other ways you might implement the many-users-to-one-club relationship - I thought of storing a "clubs" repeated property in the User that references the clubs they are members of (stores the key of the Club). You query on Users that satisfy the clubs property, and limit the results to the page size. Use a pattern like

next_page_results = User.all().filter('club =', club_key).filter('__key__ >', last_seen_key).order('key').run(limit=page_size)

to ensure that the pages are retrieved right, starting and stopping at the right place

(where last_seen_key = next_page_results[-1] for the next call)

The way you're doing it, you fetch all results every time and filter in memory. This is bad, and will cost you money.

Upvotes: 1

EsseTi
EsseTi

Reputation: 4271

Here i'm. i did some testing and i found this solution (to have the method at club/{id}/members

I created an resource container for the standard endpoints. The messages copies what the query_method has as input.

ID_RESOURCE_PAGE = endpoints.ResourceContainer(
    message_types.VoidMessage,
    id=messages.IntegerField(1, variant=messages.Variant.INT64),
    cursor=messages.StringField(2, variant=messages.Variant.STRING, required=False, default="1"),
    limit=messages.IntegerField(3, variant=messages.Variant.INT32, required=False, default=10)
)

then i created a standard @endopint.method like this

@endpoints.method(ID_RESOURCE_PAGE, User.ProtoCollection(),
                      path='club/{id}/members',
                      http_method='GET',
                      name='club.members')
    def club_memebers(self, request):
        # check if user has ownership
        club = Club.get_by_id(request.id)
        page_size = request.limit
        # convert the cursors, usually it's a token, here is page number.
        page = int(request.cursor)
        # internal check, just in case.
        if (page is None or page < 0):
            raise endpoints.BadRequestException(message="Page field must be a positive integer")
        if (page_size is None or page_size < 0 or page > 100):
            raise endpoints.BadRequestException(
                message="Page_size field must be a positive integer and cannot be greater than 100")
        # compute start and end users to retrive
        start = (page - 1) * page_size
        end = page * page_size
        # crop the list
        res_list = club.membersUser[start:end]
        # create the object
        ret = User.ToMessageCollection(res_list)
        # it's probably another page
        if (len(res_list) == page_size):
            # add next page as nextPageToken, not the best but the easy way
            ret.nextPageToken = str(page + 1)
        return ret

To note that i used User.ProtoCollection() to get the collection serialized automatically and i faked the page number into ret.nextPageToken. This last edit does not look too clean (and indeed it's not), but query works.

Still, i'm not really happy of this solution.

Upvotes: 1

Related Questions