CODEWITHSUNDEEP
javaspringspring-batch
Max Marusich
Max Marusich

Reputation: 15

Spring Batch commit-interval doesn't work and weird job behaviour

I have weird job behavior and can't understand why this is happening. I have following spring batch configuration:

<beans:bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
    <beans:property name="corePoolSize" value="50"/>
    <beans:property name="maxPoolSize" value="100"/>
    <beans:property name="queueCapacity" value="100"/>
</beans:bean>



<job id="creationFlowSaveJob">
    <step id="creationFlowCampaignSaveStep">
        <tasklet task-executor="taskExecutor"
                 throttle-limit="5">
            <chunk
                    reader="creationFlowCampaignSaveReader"
                    processor="creationFlowCampaignSaveProcessor"
                    writer="creationFlowCampaignSaveWriter"
                    commit-interval="100"
                    >
            </chunk>
        </tasklet>
        <listeners>
            <listener ref="generalStepLogger"/>
        </listeners>
    </step>
</job>

So I have step with throttle-limit = 5 and commit-interval="100" and I assume that writer will receive chunk of 100 items to write, and all chunks will be processed one by one.

But I have following flow:

If job has 4 items in flow then I see that writer receives 1 item and calling 4 times, instead of single time with 4 items. In addition to that all this 4 calls are executing concurrently, meaning at same time, which is very weird.

Another weird thing is if I remove task-executor from tasklet configuration then job starts to behave like expected. The chunk of files which is coming to writer is 4, or 100 if items amount is big and I don't have concurrent writer calls.

Can someone explain why this happening? Why taskExecutor changes job flow in this weird way and what the purpose if taskExecutor in job?

Please help to understand this, Thanks

Upvotes: 0

Views: 1275

Answers (1)

Dmitry
Dmitry

Reputation: 884

With task-executor and throttle-limit you declare multi-thread step. So you have 5 threads that concurrently read and write. If you don't need so, remove task-executor and throttle-limit from step declaration.
Also if you use multi-threading you should use thread-safe ItemReader and ItemWriter

Upvotes: 1

Related Questions