CODEWITHSUNDEEP
javaspringspring-bootspring-batch
reiley
reiley

Reputation: 3761

Spring Batch - Pass data between Processor & Writer

I've a spring batch which contains reader->processor->writer.

Data passed b/w is of type Emp:

class Emp {
    iny id;
    String name;
    EmpTypeEnum empType;    // HR, Dev, Tester, etc.

    // getters and setters
}

As a simple batch data is read from a CSV file in Reader, some processing inside Processor & and an output CSV file is written by Writer.

But apart from this output CSV file, I want to generate a secondary output file which only contains count of each EmpType, i.e. Total number of HR, Dev & Tester.

I was thinking of performing the counting within the processor only, like:

public class EmpItemProcessor implements ItemProcessor<Emp, Emp> {

    int countHr;
    int countDev;
    int countTester;


        @Override
        public Person process(final Emp emp) throws Exception {

        if (item.getEmpType.equals(EmpTypeEnum.HR) {
            countHr++;
        } else if // .....

        // other processor on emp

            return emp;
        }

}

But as you can see I can only return Emp from Processor, so how can I pass countHr, countDev, etc. from processor & use it to create secondary file?

Please suggest. If you think any other approach will be better, please suggest.

Thanks

Upvotes: 2

Views: 8039

Answers (2)

pvpkiran
pvpkiran

Reputation: 27068

You could use ItemWriteListener and JobExecutionListenerSupport for this.

  1. Define a ItemWriteListener , Which will be called after calling your writer every time.

  2. In this Listener update a counter in execution context every time

  3. Write a JobExecutionListener which will be called after the whole job is completed, where you can read the value from execution context and do further processing. 

    @Component
@JobScope
public class EmployeeWriteListener implements ItemWriteListener<Emp> {

  @Value("#{jobExecution.executionContext}")
  private ExecutionContext executionContext;


  @Override
  public void afterWrite(final List<? extends Emp> paramList) {

     final int counter =
          this.executionContext.getInt("TOTAL_EXPORTED_ITEMS", 0);
      this.executionContext.putInt("TOTAL_EXPORTED_ITEMS", counter + 1);
    }

  }
}



@Component
@JobScope
public class EmployeeNotificationListener extends JobExecutionListenerSupport  {

@Override
public void afterJob(final JobExecution jobExecution) {

  jobExecution.getExecutionContext()
      .getInt("TOTAL_EXPORTED_ITEMS")
  ...................
   }
 }

You should register these listeners when you declare your step and job .

this.jobBuilders.get("someJob").incrementer(new RunIdIncrementer()).listener(new EmployeeNotificationListener())
        .flow(this.getSomeStep()).end().build();
//instead of new(..) you should Autowire listener

public Step getSomeStep() {

  return stepBuilders.get("someStep").<X, Y>chunk(10)
      .reader(this.yourReader).processor(this.yourProcessor)
      .writer(this.yourProcessor).listener(this.EmployeeWriteListener)
      .build();
}

Upvotes: 4

gohil90
gohil90

Reputation: 517

Basically you need multple ItemWriter's to process two different writing tasks. You can easily use CompositeItemWriter which has the capability of holding list of different ItemWriter within it. And on each item it will call all it's ItemWriter.

In your case,

  1. Make two FlatFileItemWriter - one for your normal CSV output & other for your statistics.

  2. Then create CompositeItemWriter<Emp> object and add both these FlatFileItemWriter<Emp> into it using this method of it - public void setDelegates(List<ItemWriter<Emp>> delegates)

  3. Use this CompositeItemWriter as you ItemWriter in the step

So, when your CompositeItemWriter is called it will delegate to both the ItemWriter in order you have added to the list.

Job done :)

Upvotes: 1

Related Questions