CODEWITHSUNDEEP
sqldatabasespring-batch
Siva VG
Siva VG

Reputation: 45

Spring Batch - Is it suitable for this usecase?

I have a usecase that I need to evaluate and see if it is suitable for Spring Batch. I am trying to build a consolidation DB by gathering data from n different DBs by running Stored Procs (SP) on each and every DB separately, the results of the stored proc from n different DBs are consolidated in one consolidation DB, which will be further used for downstream processing. This consolidation process should run say thrice daily. I want to build a consolidation layer based on java that executes the SP in the n DBs and inserts the result in to the single consolidation DB. I am considering writing a multithreaded custom code to achieve this, as well as see if Spring batch is suitable or if there is a methodology that will completely eliminate the consolidation layer itself and directly do it via some DB methodology. Thanks for the help, pointers.

Upvotes: 0

Views: 592

Answers (2)

incomplete-co.de
incomplete-co.de

Reputation: 2137

Spring Batch has the 

org.springframework.batch.item.database.StoredProcedureItemReader

that can be used as part of a chunk processor to do what you are proposing. this would allow you to use the reader to execute the SP, then use your own processor (if required) followed by the

org.springframework.batch.item.database.JdbcBatchItemWriter

to complete the process.

if you're looking at a process that is multiple periods in a day, you might also want to consider an event-driven model and look at Spring Integration and it's 

<int-jdbc:stored-proc-inbound-channel-adapter/>

adapter. this would allow you to poll the target databases, execute the SP in question and then forward the results for persisting in your downstream database at more frequent intervals.

Upvotes: 1

Juzer Ali
Juzer Ali

Reputation: 4167

IMO this is a good use case for spring-batch. It will provide a cushion for transient errors and you can break down large operations into smaller chunks. It supports multithreaded jobs as well.

Upvotes: 0

Related Questions