CODEWITHSUNDEEP
javamultithreadingspring-batch
Kenneth Lui
Kenneth Lui

Reputation: 11

Spring batch multithreading has a default limit of 10 implicitly?

I am using TaskExecutorPartitionHandler to divide my job in to multiple PartitionStep (controlled by gridSize) and also use SimpleAsyncTaskExecutor to process asynchronously in each step (controlled by concurrencyLimit).

So far, if concurrencyLimit is large enough (let's say 50), I observe that the maximum concurrently processing of the whole batch job is limited to min(10, gridSize * 4).

With further testing, I identify the 4 in that formula can be modified by the throttleLimit() method of SimpleStepBuilder.

However, even if I configured gridSize and throttleLimit to some larger values, the overall concurrency never go over 10. Does anyone know where I can configure that limit?

p.s. I am configuring the job programmatically, not with XML.

Upvotes: 1

Views: 1178

Answers (1)

ACH
ACH

Reputation: 195

Bit late but try to check if you are using HikariCP which has a default 10 simultaneous DB connexions limit

Just faced similar issue and resolved by increasing spring.datasource.hikari.maximum-pool-size

See: Number of parallel threads processing is capped to 10 with Spring Batch

Upvotes: 1

Related Questions