CODEWITHSUNDEEP
javaspring-batch
Terry Deng
Terry Deng

Reputation: 951

Spring Batch Reader Parameter Issue

Currently I need to implement the following steps with Spring batch:

  1. Read table data from data source A;
  2. Read table data from data source B based on the column values I get from data source A at step 1 as search criteria;
  3. Write what I get at step 2 to some other place

Technically I have no problem dealing with step 1 and step 3, but anyone could advise how to tackle step 2? I understand that after step 1 I can get a rowMapper class that maps each row of data to my domain object, in this case how to pass the column values (domain object attributes) as the parameters to step 2?

Upvotes: 0

Views: 2065

Answers (2)

M. Deinum
M. Deinum

Reputation: 125292

Spring Batch Chunk Processing

As I tried to explain in the comments (and in the link to the documentation). Use a chunk oriented step. Your sequence corresponds to the following

  1. ItemReader
  2. ItemProcessor
  3. ItemWriter

For the reader you could use a JdbcCursorItemReader together with a RowMapper to convert the result into an object. In the ItemProcessor you use a JdbcTemplate with a query and use the incoming object to add the parameters to the query, together with another RowMapper this will convert the result into an object. This object is passed to the ItemWriter which stores the object you could use a JdbcBatchItemWriter for that.

Depending on your needs for step 2/3 you could try to create a custom writer which does the processing (reading/updating) in a single query (this could be faster then reading, constructing objects and writing again).

Upvotes: 2

Karthik Prasad
Karthik Prasad

Reputation: 10034

Why don't you create Staging table in Datasource B( I assume its different database). And tweek your query. Hence your step would be

  1. Step1: Read the data from DataSource A and write to staging table of Datasource B.
  2. Step2: Read the data from Datasouce B and write to file/some other place. (Tweek your select statement fetch in step2 to meet your conditions.

Upvotes: 1

Related Questions