Making a item reader to return a list instead single object - Spring batch

Question

Question is : How to make an Item reader in spring batch to deliver a list instead of a single object.

I have searched across, some answers are to modify the item reader to return list of objects and changing item processor to accept a list as input.

How to do/code the item reader ?

Answer 1

You can use setDelegate() to return the List to the ItemReader.

@Configuration
public class GeneralFileReader {

  @Bean
  @StepScope
  public MultiLinePeekableReader reader( @Value(FileProcessingConstants.FILENAME_JOB_PARAM)
          final String fileName) {
    FlatFileItemReader<FileStructure> itemReader = new FlatFileItemReader<>();
    final Resource resource = applicationContext.getResource(gcsLocationOfFile);
    itemReader.setResource(resource);
    itemReader.setName("FileReader : " + fileName);
    itemReader.setLineMapper(lineMapper());
    itemReader.setStrict(true);
    MultiLinePeekableReader multiLinePeekableReader = new MultiLinePeekableReader(fileName);
    multiLinePeekableReader.setDelegate(itemReader);
    return multiLinePeekableReader;
  }

  private LineMapper<FileStructure> lineMapper() {
    DefaultLineMapper<FileStructure> lineMapper = new DefaultLineMapper<>();
    DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer();
    lineTokenizer.setDelimiter(fileProcessingConfiguration.inputFileDelimiter());
    lineTokenizer.setNames(
        INPUT_FILE_FIELD_DATE,
        INPUT_FILE_FIELD_NUMBER);

    BeanWrapperFieldSetMapper<FileStructure> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
    fieldSetMapper.setTargetType(Expected.class);

    lineMapper.setLineTokenizer(lineTokenizer);
    lineMapper.setFieldSetMapper(fieldSetMapper);
    return lineMapper;
  }
}

and in the MultiLinePeekableReader you can set achieve the list by something:

public class MultiLinePeekableReader implements ItemReader<List<ExpectedClass>>, ItemStream {

  private SingleItemPeekableItemReader<ExpectedClass> delegate;

  @Override
  @SneakyThrows
  @Bean
  @StepScope
  public synchronized List<ExpectedClass> read() {
    List<Object> records = null;

    for (ExpectedClass line, int loop=0; (line = this.delegate.read()) != null && loop<10; loop++) {
        //add logic here.
        if(records==null){
           records=new ArrayList<>();
        }  

        records.add(line);
        ExpectedClass nextLine = this.delegate.peek();
        if (nextLine == null) {
            return records;
        }
      }
    }
    return records;
  }

  @Override
  public void close() throws ItemStreamException {
    this.delegate.close();
  }

  @Override
  public void open(ExecutionContext executionContext) throws ItemStreamException {
    this.delegate.open(executionContext);
  }

  @Override
  public void update(ExecutionContext executionContext) throws ItemStreamException {
    this.delegate.update(executionContext);
  }

  public void setDelegate(FlatFileItemReader<FileStructure> delegate) {
    this.delegate = new SingleItemPeekableItemReader<>();
    this.delegate.setDelegate(delegate);
  }
}

Answer 2

take a look at the official spring batch documentation for itemReader

public interface ItemReader<T> {

    T read() throws Exception, UnexpectedInputException, ParseException;

}
// so it is as easy as
public class ReturnsListReader implements ItemReader<List<?>> {
   public List<?> read() throws Exception {
      // ... reader logic
   }
}

the processor works the same

public class FooProcessor implements ItemProcessor<List<?>, List<?>> {

    @Override
    public List<?> process(List<?> item) throws Exception {
        // ... logic
    }

}

instead of returning a list, the processor can return anything e.g. a String

public class FooProcessor implements ItemProcessor<List<?>, String> {

    @Override
    public String process(List<?> item) throws Exception {
        // ... logic
    }

}