CODEWITHSUNDEEP
springspring-batch
vishal
vishal

Reputation: 4083

Can spring batch multi-threaded step be used safely if number of items in file are very less?

I read in documentation, multi-threaded step is not safe to be used as many of the ItemReaders and Writers are not thread safe.

I am using FlatFileItemReader to read and process items.

In case of the file where there are huge number of items to be processed, I am using remote partitioning.

But some of the steps has input file with just 2-3 items (they are just market ids e.g eu, gb, etc). I will be adding few more markets. I need to run some commands which takes these market ids as input. I want to run commands in parallel for all of them.

Is multithreaded step safe to use for such use case even though I am using FlatFileItemReader ? or I should go for remote partitioning (there is not much data to be partitioned) ?

Also if I use multi-threaded step, will it run properly I launch multiple instances of same job with different parameters e.g. different dates ?

Upvotes: 0

Views: 750

Answers (1)

Michael Minella
Michael Minella

Reputation: 21463

The FlatFileItemReader is not thread safe in that it's state is based on the number of rows in the file have been read. When using it with multiple threads, that number gets overwritten and so there isn't a way to know what has and has not been read on a restart. If restartability is not an issue (you're ok with starting from the beginning if the job fails), then you can use multiple threads in a step with the FlatFileItemReader.

Upvotes: 3

Related Questions