CODEWITHSUNDEEP
springspring-batch
Pratik Shelar
Pratik Shelar

Reputation: 3212

Impact of increasing the commitInterval in SpringBatch

I have written a spring batch which has commitInterval of 50. The itemReader which I am using has fetchSize of 200. So according to my understanding the reader will fetch 200 records but only chunks of 50 would be commited. So for each fetch of 200 records 4 commits of 50 would be performed. Am I correct?

My issue is that we have many spring batches configured and there could be a scenario where multiple batches are running at same time. If I increase the commitInterval to 200 will it affect the performance of other batches which have lesser commitIntervals?

Please advise.

Upvotes: 2

Views: 2183

Answers (1)

incomplete-co.de
incomplete-co.de

Reputation: 2137

  1. yes - you're right, fetchSize will bring in 200 records and commit-interval will 'commit' in 50 record blocks.

  2. larger commit intervals can roughly translate to larger transaction 'cursors' to commit. that is, you will gain some performance in the transaction overhead being reduced (about 1/4 if set to 200).

the 'risk' in doing so is that you lose more 'work' should an exception occur in the commit interval. (instead of losing a maximum of 49 records on a failure, you could lose up to 199 and have to do them again)

depending on your architectures exposure to failures (i.e. how safe and stable is your datasource connection for instance) will govern how risky committing 200 versus 50 records at a time will be. i would be in favor of matching your fetchSize to your commit-interval as without a retry manager, you lose the fetchSize when a failure occurs on commit.

Upvotes: 3

Related Questions