CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaaws-api-gateway
rickythefox
rickythefox

Reputation: 6851

Counting AWS lambda calls and segmenting data per api key

Customers (around 1000) sign up to my service and receive a customer unique api key. They then use the key when calling a AWS lambda function through AWS api gateway in to access data in DynamoDb.

Requirement 1: The customers get billed by the number of api calls, so I have to be able to count those. AWS only provides metrics for total number of api calls per lambda so I have a few options:

  1. At every api hit increment a counter in DynamoDB.
  2. At every api hit enqueue a message in SQS, receive it in "hit counter" lambda and increment a counter in DynamoDB.
  3. Deploy a separate lambda for each customer. Use AWS built-in call counter.

Requirement 2: The data that the lambda can access is unique for each customer and thus dependent on the api key provided. To enable this I also have a number of options:

  1. Store the required api key together with the data that the customer has the right to access.
  2. Deploy a separate lambda for each customer. Use api gateway to protect it with a key.
  3. Create a separate endpoint in api gateway for each customer, protect it with the api key.

None of the options above seem like a good way to design the solution. Is there a canonical way of doing this? If not, which of the options above is the best? Have I missed an obvious solution due to my unfamiliarity with AWS?

Upvotes: 4

Views: 1834

Answers (1)

Thales Minussi
Thales Minussi

Reputation: 7235

I will try to break your problems down with my experience, but maybe Michael - Sqlbot or John Rotenstein may be able to give more appropriate answers.

Requirement 1

1) This sounds like a good approach. I don't see anything critical here.

2) This, IMHO, is the best out of the 3. It will decouple data access from the billing service, which is a great thing in a Microservices world.

3) This is not scalable. Imagine your system grows and you end up with 10K Lambda functions. Not only you'll have to build a very reliable mechanism to automate this process, but also you'll need to monitor 10K different things (imagine CloudWatch logs, API Gateway, etc), not to mention you'll have 10 thousand functions with exactly the same code (client specific parameters apart). I wouldn't even think about this one.

Requirement 2

1) It could work and it fits nicely in the DynamoDB model of doing things: store as much data as you can in a unique table, so you can fetch everything in one go. From what I see, you could even use this ApiKey as your partition key and, for the sake of simplicity for this answer, store the client's data as JSON in a column named data. Since your query only needs to query by the ApiKey, storing a JSON in DynamoDB won't hurt (do keep in mind, however, that if you need to query by any of its JSON attributes than you're in bad shoes, since DynamoDB's query capabilities are very limited)

2) No, because of Requirement 1.3

3) No, because of the above.

If you still need to store the ApiKey in a different table so you can run different analysis and keep a finer grained control over the client's calls, access, billing and etc., that's not a problem either, just make sure you duplicate your ApiKey on your ClientData table instead of creating a FK (DynamoDB doesn't support FKs, so you'd need to manage these constraints yourself). Duplication is just fine in a NoSQL world.

Your use case is clearly a Multi-Tenancy one, so I'd also recommend you to read Multi-Tenant Storage with Amazon DynamoDB which will give you some more insights and broaden your options a little bit. Multi-Tenancy is not an easy task and can give you lots of headaches if not implemented correctly. I think this is why AWS has also prepared this nice read for us :)

Happy to continue this on the comments section in case you have more info to share

Hope this helps!

Upvotes: 3

Related Questions