Spring Batch ItemReader list processed only once

Question

I'm trying to create a Spring Batch job using a ListItemReader<String>, ItemProcessor<String, String> and ItemWriter<String>.

The XML looks like the following,

<job id="sourceJob" xmlns="http://www.springframework.org/schema/batch">
    <step id="step1" next="step2">
        <tasklet>
            <chunk reader="svnSourceItemReader" 
                processor="metadataItemProcessor" 
                writer="metadataItemWriter" 
                commit-interval="1" />
        </tasklet>
    </step>
    <step id="step2">
        <tasklet ref="lastRevisionLoggerTasklet"></tasklet>
    </step>
</job>

<bean id="svnSourceItemReader" 
        class="com.example.repository.batch.SvnSourceItemReader" 
        scope="prototype">
    <constructor-arg index="0">
        <list>
            <value>doc1.xkbml</value>
            <value>doc2.xkbml</value>
            <value>doc3.xkbml</value>
        </list>
    </constructor-arg>
</bean>

<bean id="metadataItemProcessor" 
        class="com.example.repository.batch.MetadataItemProcessor" 
        scope="prototype" />

<bean id="metadataItemWriter" 
        class="com.example.repository.batch.MetadataItemWriter" 
        scope="prototype" />

The reader, processor and writer are vanilla,

public class SvnSourceItemReader extends ListItemReader<String> {

    public SvnSourceItemReader(List<String> list) {
        super(list);
        System.out.println("Reading data list " + list);
    }

    @Override
    public String read() {
        String out = (String) super.read();
        System.out.println("Reading data " + out);
        return out;
    }

}

public class MetadataItemProcessor implements ItemProcessor<String, String> {

    @Override
    public String process(String i) throws Exception {
        System.out.println("Processing " + i + " : documentId " + documentId);
        return i;
    }

}

public class MetadataItemWriter implements ItemWriter<String> {

    @Override
    public void write(List<? extends String> list) throws Exception {
        System.out.println("Writing " + list);
    }

}

The job is started like this, but on a schedule of every 10 seconds.

long nanoBits = System.nanoTime() % 1000000L;
if (nanoBits < 0) {
    nanoBits *= -1;
}
String dateParam = new Date().toString() + System.currentTimeMillis() 
        + "." + nanoBits;
param = new JobParametersBuilder().addString("date", dateParam)
        .toJobParameters();
JobExecution execution = jobLauncher.run(job, param);

When the application starts, I see it read, process and write each of the three items in the list passed to the reader.

Reading data doc1.xkbml
Processing doc1.xkbml : documentId doc1
Writing [doc1.xkbml]
Reading data doc2.xkbml
Processing doc2.xkbml : documentId doc2
Writing [doc2.xkbml]
Reading data doc3.xkbml
Processing doc3.xkbml : documentId doc3
Writing [doc3.xkbml]

Because this sourceJob is on a scheduled timer, every 10 seconds I expected to see that list processed, but instead I see on all subsequent runs.

Reading data null

Does anyone know why this is happening? I'm new to Spring Batch and just can't get my hands around the issue.

Thanks /w

Answer 1

Just an additional information: you can also use JavaConfig instead of the xml config file, and annotate the reader bean declaration with @StepConfig.

ex:

@Configuration
@EnableBatchProcessing
public class MyConfig {
...

    @Bean
    @StepScope
    public ItemReader<HeadingBreakevenAssociation> readerHeadingBreakevenAssociationList(){
         ItemReader<Person> itemReader = new ListItemReader<Person>(myList);

          return itemReader;
    }

}

Answer 2

The problem is that you marked your reader as scope="prototype". It should be scope="step".

In Spring-batch there are only two scopes: singleton (the default) and step.

From the javadoc:

StepScope:
Scope for step context. Objects in this scope use the Spring container as an object factory, so there is only one instance of such a bean per executing step. All objects in this scope are (no need to decorate the bean definitions).

and

Using a scope of Step is required in order to use late binding since the bean cannot actually be instantiated until the Step starts, which allows the attributes to be found.

During the Spring context startup look at your log and you will see this line:

INFO: Done executing SQL script from class path resource [org/springframework/batch/core/schema-hsqldb.sql] in 9 ms.
Reading data list [doc1.xkbml, doc2.xkbml, doc3.xkbml]

as you can see your reader has already been created and managed as a singleton; dynamic beans in spring-batch context should be managed with the special step scope so that Spring will create a fresh copy of the bean every time a step is executed.

In your reader, ListItemReader.read() is written as:

public T read() {
  if (!list.isEmpty()) {
    return list.remove(0);
  }
  return null;
}

In each read items are removed from original list! The reader is constructed once and, on second job execution, the list is empty!