CODEWITHSUNDEEP
amazon-dynamodb
Leff
Leff

Reputation: 1298

Dynamodb sparse indexes implementation

I am new to dynamodb, and I came across sparse indexes. I think they fit for what I need, but I am not completely sure how to implement them. In my case I have a table with a Post entity that has the following fields, it looks like this:

post_id <string> | user_id <string> | tags <string[]> | public <boolean>| other post data attributes...

The queries that I need to do are:

  1. Get all posts that are marked public
  2. Get posts filtered by tags
  3. Get all posts filtered by user, both public and not public
  4. Get a single post

For case of getting all public posts I think sparse indexes could work. I could set the public attribute only to the entities that are marked with public. But, how does a query look like then?

I am not even sure if I have set up the DB correctly. I am using serverless framework and this is what I have come up, but not sure if that is good.

PostsDynamoDBTable:
      Type: 'AWS::DynamoDB::Table'
      Properties:
        AttributeDefinitions:
          - AttributeName: postId
            AttributeType: S
          - AttributeName: userId
            AttributeType: S
          - AttributeName: createdAt
            AttributeType: S
        KeySchema:
          - AttributeName: postId
            KeyType: HASH
          - AttributeName: userId
            KeyType: RANGE
        BillingMode: PAY_PER_REQUEST
        TableName: ${self:provider.environment.POSTS_TABLE}
        GlobalSecondaryIndexes:
          - IndexName: ${self:provider.environment.USER_ID_INDEX}
            KeySchema:
              - AttributeName: userId
                KeyType: HASH
              - AttributeName: public
                KeyType: RANGE
            Projection:
              ProjectionType: ALL

Upvotes: 0

Views: 1223

Answers (1)

Aaron Stuyvenberg
Aaron Stuyvenberg

Reputation: 3777

A sparse index could work if you only want a subset of attributes, you'd want to use ProjectionType: INCLUDES to include the non-key attributes into the sparse index (in your case: public attributes). It's important to note that the only attributes you can access in a query to a sparse index are the attributes you explicitly include.

Firstly, you'd need to declare those public attributes in your Attribute Definitions.

Say for example, one of the public attributes is userName. You'd want to add:

 - AttributeName: userName
   AttributeType: S

Then, in the GlobalSecondaryIndexes block:

GlobalSecondaryIndexes:
  - IndexName: byUserIdAndPublic // You can name this whatever you'd like
    KeySchema:
      - AttributeName: userId
        KeyType: HASH
      - AttributeName: public
        KeyType: RANGE
      Projection:
        NonKeyAttributes:
        - userName
      ProjectionType: INCLUDE

Then you simply query that index specifically - in the response, you'll get back the userName (no special query changes are required, except to specify using this index).

If you need all attributes for each post, you'd want to use ProjectionType: ALL, (and then just remove the NonKeyAttributes bit)

Here's an example nodejs method that would filter posts by public (public being a boolean passed into the method):

const listByUserIdAndPublic = async (userId, public) => {
  const params = {
    TableName: tableName,
    IndexName: 'byUserIdAndPublic',
    KeyConditionExpression: '#public = :public AND #userId = :userId',
    ExpressionAttributeNames: {
      '#userId': 'userId',
      '#public': 'public'
    },
    ExpressionAttributeValues: {
      ':public': public,
      ':userId': userId
    },
  };
  const response = await documentClient.query(params).promise();
  return response.Items;
};

Based on your question:

Get all posts that are marked public

I think you'd want to use a full index (ProjectionType: ALL), because I'd imagine you want all attributes of a post to be indexed.

Upvotes: 2

Related Questions