CODEWITHSUNDEEP
node.jsamazon-web-servicesamazon-dynamodbleaderboard
Rabin Mallilck
Rabin Mallilck

Reputation: 471

Top 10 scores for game leaderboard dynamoDB nodejs sdk query

I have a dynamodb which has following fields -

id (primary key) (string)
name (user name) (string)
score (sort key) (number)
player (type - regular/new etc.) (string)

I'm just trying to get top 10 scores from the DB. Tried several queries following the documentation by AWS https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB.html#query-property

Here is my query code -

const ddb = new AWS.DynamoDB.DocumentClient({apiVersion: '2012-08-10'});

function getItems(){
  var params = {
  ExpressionAttributeValues: {
   ":v1": {
     S: "regular"
    }
  }, 
  KeyConditionExpression: "player = :v1", 
  TableName: "cloudtag_result"
 };
 return ddb.scan(params).promise();
}

but it's giving me following error

2021-08-21T08:36:43.092Z    1bbd5490-4248-4f06-94ea-724e6d23dac5    ERROR   ValidationException: Query condition missed key schema element: id
    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:52:27)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:688:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:690:12)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
  code: 'ValidationException',
  time: 2021-08-21T08:36:43.031Z,
  requestId: 'JH91VON2KGVNP7HF41VCBELTUVVV4KQNSO5AEMVJF66Q9ASUAAJG',
  statusCode: 400,
  retryable: false,
  retryDelay: 47.72972319109784
}

Upvotes: 0

Views: 789

Answers (2)

ShengHow95
ShengHow95

Reputation: 246

You could create a Global Secondary Index (GSI) for this.

Take player as your GSI Partition Key and score as your GSI Sort Key, you could actually perform query against this GSI by providing the value for player and use scanIndexForward (true/false) for getting the data in ascending or descending order based on the GSI Sort Key.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html

However, when your data grows bigger, you would probably hit Hot Partition problem on your GSI which is due to the GSI Partition Key that is too much concentrated on a specify key such as regular. Therefore, it would be better if you could start doing partition key sharding which you can add a suffix to your player such as regular#1, regular#2... regular#n. In this way, you would be able to avoid from Hot Partition problem.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-sharding.html

Again, if you adopt this solution, you would need to do a n times of queries on the GSI to get the top few scorers in each partition due to the partition key sharding and only then you will use code to do the sorting.

Upvotes: 1

August Lilleaas
August Lilleaas

Reputation: 54603

DynamoDB has the fundamental limitation that you absolutely have to specify the partition key (or primary key as you call it) in all queries. And it has to be an exact match for what's in the table, no "fuzzy" querying allowed.

You query on player = .., and you don't specify the id. There's no way to make that work in dynamo, you need to specify the primary key.

If your access patterns requires you to be able to look up by player name, you need to have the player name in a PK or a SK, or else you need to do the filtering in your app instead of on the dynamodb server.

You probably want what's called a "single table design". This talk by Rich Houlihan comes highly recommended for anyone doing data modelling with dynamo :)

https://www.youtube.com/watch?v=HaEPXoXVf2k

Upvotes: 0

Related Questions