CODEWITHSUNDEEP
c#redisstackexchange.redis
Umamaheswaran
Umamaheswaran

Reputation: 3878

How to provide sort functionality using redis as the secondary indexes

Currently I'm working on a project where we need to build our secondary indexes on top of redis. Since the actual datastore is going to be a blob database like S3/Azure Blob.

So far we are able to implement our own schema manager, Create/Update/Delete for indexes whenever data is changed. And the indexes are stored as SortedSet in Redis

query.Where<Employee>(emp => emp.JoinedOn >= startDate 
               && emp.JoinedOn <= endDate 
               && emp.Salary >= 150000);

We have also implemented querying logic using the C# expression. Where we built an expression parser to go through the expression lambda and run each binary expression and combine the results.

Once we run this query we will have list of keys, which we can go and fetch from S3/Azure blob.

The Problem

Now the problem is if we want to support sorting we have to fetch the entire index data and intersect the result with the result that we got after querying.

private async Task<string[]> SortResults(QueryModel queryModel, string sortIndex, Stack<List<string>> resultStore)
        {
            var order = queryModel.SortInfo.Direction == SortDirection.Ascending
                ? Order.Ascending
                : Order.Descending;
            var sortedSet = await cacheProvider.SortedSetGetRangeByScoreAsync(sortIndex, 
                double.NegativeInfinity, 
                double.PositiveInfinity, 
                Exclude.None, 
                order);
            var ids = resultStore.Any() ? resultStore.Pop().ToArray() : null;
            return (string[]) sortedSet.Intersect(ids ?? Array.Empty<string>());
        }

Is there a way we could avoid querying the entire index to support this?

Upvotes: 0

Views: 469

Answers (1)

sazzad
sazzad

Reputation: 6267

Is there a way we could avoid querying the entire index to support this?

Yes and no.

You can't achieve this using only plain Redis.

But you can (and should) consider RediSearch - Redis Secondary Index & Query Engine which is very well suited for your requirement and beyond. This is a module built on top of Redis and maintained by Redis Labs itself.

Upvotes: 1

Related Questions