CODEWITHSUNDEEP
javaspringspring-mvcspring-integration
bmylavar
bmylavar

Reputation: 61

Scaling a Spring Integration web application

There is this stateless REST application/API written and being maintained by me using Spring Integration API with the following underlying concepts working hand-in-hand:

1) Inbound HTTP gateway as the RESTful entrypoint 2) A handful of Service Activators, Routers, Channels and Transformers 3) A Splitter (and an Aggregator) with the former subscribed to a channel which in turn, has a task executor wired-in comprising a thread pool of size 100 for parallelised execution of the split(ted) messages

The application is performing seamlessly so far - as the next step, my attempt is to scale this application to handle a higher number of requests in order to accommodate a worst case situation where all 100 threads in the pool are occupied at the exact same time.

Please note that the behaviour of the service is always meant to be synchronous (this is a business need) and there are times when the service can be a slightly long-running one. The worst-case roundtrip is ~15 seconds and the best case is ~2 seconds, both of which are within acceptable limits for the business team.

The application server at hand is WebSphere 8.5 in a multi-instance clustered environment and there is a provision to grow the size of the cluster as well as the power of each instance in terms of memory and processor cores.

That said, I am exploring ways to solve the problem of scaling the application within the implementation layer and these are a couple of ways I could think of:

1) Increase the size of the task executor thread pool by many times, say, to 1000 or 10000 instead of 100 to accommodate a higher number of parallel requests. 2) Keep the size of the task executor thread pool intact and instead, scale-up by using some Spring code to convert the single application context into a pool of contexts so that each request can grab one that is available and every context has full access to the thread pool. Example: A pool of 250 application contexts with each context having a thread pool of size 100, facilitating a total of 250 × 100 = 25000 threads in parallel.

The 2nd approach may lead to high memory consumption so I am thinking if I should start with approach 1). However, what I am not sure of is if either of the approaches is practical in the long run.

Can anyone kindly throw some light? Thanks in advance for your time.

Sincerely, Bharath

Upvotes: 2

Views: 664

Answers (1)

Rafa
Rafa

Reputation: 2027

In my experience, it is very easy to hit a road block when scaling up. In contrast, scaling out is more flexible but adds complexity to the system.

The application server at hand is WebSphere 8.5 in a multi-instance clustered environment and there is a provision to grow the size of the cluster as well as the power of each instance in terms of memory and processor cores.

I would continue in this direction (scaling out by adding instances to the cluster), if possible I would add a load balance mechanism in front of it. Start by distributing the load randomly and enhance by distributing the load by "free threads in the instance's pool".

Moreover, identify the heavier portions of the systems and evaluate if you would gain anything by migrating them to their own dedicated services.

Please note that the behaviour of the service is always meant to be synchronous (this is a business need) and there are times when the service can be a slightly long-running one.

The statement above raises some eyebrows. I understand when the business says "only return the results when everything is done". If that is the case then this system would benefit a lot if you could change the paradigm from a synchronous request/response to an Observer Pattern.

Upvotes: 1

Related Questions