CODEWITHSUNDEEP
spring-xd
Peter Rietzler
Peter Rietzler

Reputation: 481

How to configure the number of parallel job executions

It looks like that the number of parallel job executions is limited. My observation is that the Spring XD runtime (singlenode mode) executes at most 10 jobs at the same time.

I've created 15 jobs (wait1, wait2, ...) that simply wait for a minute and executed them right after each other (job launch wait1, job launch wait2, ...).

I've got a lot of jobs that block because they are waiting for other servers to respond, so I'd like to increase the number of executing jobs. Couldn't find anything in the documentation and couldn't find anything in servers.yml

Upvotes: 1

Views: 130

Answers (1)

Gary Russell
Gary Russell

Reputation: 174719

When running in singlenode mode with a local transport, batch jobs run on the Spring Integration TaskScheduler which defaults to 10 threads, per the documentation.

To override this config, we need to add a property to servers.yml; please open a JIRA issue.

To override this, a taskScheduler bean needs to be added to the global context.

In the meantime, if you use a different transport (redis or rabbit), you won't have this limitation.

EDIT:

To increase the thread count for the scheduler:

  • create a subdirectory META-INF in xd/config
  • create a file spring.integration.properties in that directory (xd/config/META-INF)
  • add a property: spring.integraton.taskScheduler.poolSize=30
  • restart xd-singlenode

(where 30 is a little larger than the number of concurrent batch jobs you want to run).

Upvotes: 2

Related Questions