CODEWITHSUNDEEP
architecturedevops
Slx.x
Slx.x

Reputation: 57

How would you scale api gateway service?

I'm currently designing a service that would enable my clients to create and deploy their APIs not unlike AWS Api gateway. It will be used as API entry point and routed to multiple backends. The logic of such gateway is clear to me, however the overall architecture is still not.

In accordance with modern SaaS techniques and rules I'll need to create a fail-resistant scalable solution which usually takes some form of parallelism in servicing requests (at least as for APIs in general).

This gateway is able to create and deploy client APIs (set of endpoints) which are injected in the main routing tree of an application. For instance user1 create API which 3 endpoints, I inject them into the tree as /user1/stage/xx[1,2,3]. Then another one does the same. Gateway controls deployment and servicing of their endpoints and stores all their data in memory for fast routing.

Here comes scalability. I need to build gateway in such manner that the solution wouldn't fail, would support scaling up and down according to its needs however I don't really see how can I distribute single application logic like that to many instances.

Two approaches come to mind and they both have their weaknesses:

  1. Separate services based on a number of apis it processes. i.e. instance1 is servicing 100 API so it's time to create a new one. This would need some routing that main balancer would know where to send remote request - which instance is handling what. And failure of one of instances would bring 100 APIs down

  2. Separate only instances pertaining all the logic and data in one place. Bring up as many instances as needed, load the same data to them and task balancer to route remote requests according to balancing logic. This would require all instances to bring ALL the APIs and some sort of control point to deploy and bring down client APIs on all instances but this accomplishes the load scalability. However if there will be a LOT of APIs each instance will have to deploy them all in its memory (handlers, routing tables etc).

Am I missing something? How would you go about this problem? What's the best way to build scalable and fail-safe api gateway such as this?

Upvotes: 1

Views: 3217

Answers (1)

Muhammad Imran
Muhammad Imran

Reputation: 58

Well, its late, but I did it using containerization, have wrote a custom service for API Gateway & then deploy it as a container in Kubernetes cluster. K8s took care of the scaling after by replicating pods.

Upvotes: 1

Related Questions