Cognito ListUsers continues forever

Question

Hey I wrote some code to download all the users in my Cognito user pool however it seems to go on forever. I'm quite sure I'm using the pagination token correctly. Cognito's UI estimates I have ~10,000 users however I put a breakpoint minutes into my loop and my List has over 50,000 which doesn't make sense.

Help much appreciated!

        using (AmazonCognitoIdentityProviderClient provider = AuthorizedClient())
        {
            try
            {
                List<UserType> users = new List<UserType>();
                bool continueListing = true;

                ListUsersResponse response = provider.ListUsers(
                    new ListUsersRequest
                    {
                        UserPoolId = UserPoolId,
                        Limit = 60,
                        AttributesToGet = new List<string>
                        {
                            "email"
                        }
                    });

                users.AddRange(response.Users);
                string paginationToken = response.PaginationToken;
                while (continueListing)
                {
                    response = provider.ListUsers(
                        new ListUsersRequest
                        {
                            UserPoolId = UserPoolId,
                            Limit = 60,
                            PaginationToken = paginationToken,
                            AttributesToGet = new List<string>
                            {
                                "email"
                            }
                        });

                    if (response.Users.Count < 60)
                    {
                        continueListing = false;
                    }
                    else
                    {
                        paginationToken = response.PaginationToken;
                        users.AddRange(response.Users);
                    }
                }

                return users;
            }
            catch (Exception)
            {
                //_session.NotifyUser(Notification.GeneralError());
                //_logger.LogError(e.Message);
            }

            return null;
        }

Answer 1

I ended up getting it working using a rate gate. I think when you spam requests something gets jumbled up in the HTTP universe and you start getting funny responses. Anyway if you exceed 5 ListUsers API calls per second you'll get a RateExceedException and it won't work regardless, so I needed a rate gate anyway.

This is the 'RateGate' I used and it works really well. I just set it to limit my loop to 4 iterations per second (one less than the maximum).

https://github.com/Danthar/RateLimiting

Answer 2

Do not list your users from cognito! Your going to start hitting their api limits. Poll your database for your users, you have more control.