CODEWITHSUNDEEP
javaspringspring-bootspring-batch
S-Wing
S-Wing

Reputation: 579

Process all lines at the same time with spring batch

I'm writing a batch program, using Spring batch, that reads from Db and then writes the information to a csv file. I've already used this framework before, but all the other programs that I wrote had the same structure:
1- read a line from db;
2- process the line individually;
3- write the line to the file;
But this new program has a different logic: I nedd to read lines from the Db, the I have to process all of them ( ex. if I found multiple lines with the same value in field A, I need to print only one line with field B that is the sum of the field B of the other lines ), and then I have to write. Is it possible to do something like this with Spring batch ? ( any example that I found works on single line ).
Thanks.

Upvotes: 0

Views: 1474

Answers (2)

Mahmoud Ben Hassine
Mahmoud Ben Hassine

Reputation: 31600

Aggregate functions require the entire data set to be processed in order to be calculated. The chunk-oriented processing model is not very friendly to implementing this kind of functions since data is processed in separate chunks.

In your case, since data is read from a database, I would leave the grouping/summing to the database with something like select sum(fieldB) ... group by fieldB. Databases are well optimized for this kind of tasks and you should end up reading a single record for each group.

Upvotes: 0

Marco Behler
Marco Behler

Reputation: 3724

Instead of working with chunks (readers, writer, processors), Spring Batch can also work with Tasklets. A Tasklet is essentially one method, that does everything.

public class YourTasklet implements Tasklet {


    public RepeatStatus execute(StepContribution contribution,
                                ChunkContext chunkContext) throws Exception {


        // TODO do everything you want here 
        return RepeatStatus.FINISHED;
    }


}

You might want to read up on tasklets in the official documentation.

Upvotes: 1

Related Questions