Spring Batch - File to be read is not available

Question

I have a Spring batch job with a first step that reads a file and writes to a database. If the file does not exist, I want the system to wait x minutes and then retry the step up to y number of attempts. How can I do this?

Details

The ItemReader is as follows:

 @Bean
    public ItemReader<Car> reader(LineMapper<Car> lineMapper, File file) {
        FlatFileItemReader<Car> flatFileItemReader = new FlatFileItemReader<Car>();
        flatFileItemReader.setResource(new FileSystemResource(file));
        final int NUMBER_OF_HEADER_LINES = 1;
        flatFileItemReader.setLinesToSkip(NUMBER_OF_HEADER_LINES);
        flatFileItemReader.setLineMapper(lineMapper);
        return flatFileItemReader;
    }

Currently, if the file does not exist, the following exception is thrown and the job fails.

2016-08-05 09:30:10.277 ERROR 7668 --- [           main] o.s.batch.core.step.AbstractStep         : Encountered an error executing step stepLoadFile in job testJob

org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:147) ~[spring-batch-infrastructure-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96) ~[spring-batch-infrastructure-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:310) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:197) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_91]
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_91]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_91]
    at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_91]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) [spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) [spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) [spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
    at com.sun.proxy.$Proxy78.run(Unknown Source) [na:na]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:215) [spring-boot-autoconfigure-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:232) [spring-boot-autoconfigure-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:124) [spring-boot-autoconfigure-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:118) [spring-boot-autoconfigure-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:804) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:788) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:775) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.doRun(SpringApplication.java:366) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:305) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1124) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1113) [spring-boot-1.3.1.RELEASE.jar:1.3.1.RELEASE]
    at com.myorg.LoadApplication.main(LoadApplication.java:15) [bin/:na]
Caused by: java.lang.IllegalStateException: Input resource must exist (reader is in 'strict' mode): file [C:\test\TestData.csv]
    at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:251) ~[spring-batch-infrastructure-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:144) ~[spring-batch-infrastructure-3.0.6.RELEASE.jar:3.0.6.RELEASE]
    ... 33 common frames omitted

com.myorg.LoadApplication code starts the batch application and is:

@SpringBootApplication
public class LoadApplication {

    public static void main(String[] args) {
        SpringApplication.run(LoadApplication.class, args);
    }

}

Update

After reading dimitrisli answer, I tried this in my @Configuration class (Note: I also added @EnableRetry to the configuration class):

@Bean
public ItemReader<Test> reader(LineMapper<Test> lineMapper, ApplicationProperties properties) {
    FlatFileItemReader<Test> flatFileItemReader = new FlatFileItemReader<Test>() {
        @Override
        @Retryable(value = {ItemStreamException.class}, maxAttempts=5)
        public void open(ExecutionContext executionContext) throws ItemStreamException {
            super.open(executionContext);
        }

        @Override
        @Retryable(maxAttempts=5)
        public Holding read() throws UnexpectedInputException, ParseException, Exception {
            return super.read();
        }

    };
    flatFileItemReader.setResource(new FileSystemResource(properties.getLoadFile()));
    flatFileItemReader.setLineMapper(lineMapper);
    return flatFileItemReader;
}

ItemStreamException is thrown and the application exits without retrying. How can I get it to retry?

Answer 1

By using Spring Retry, the standalone spinoff project from Spring Batch: Assuming the Exception potentially thrown, that you would want to recover from and retry, say 5 times, is ItemStreamException then you'll have to decorate your method that potentially throws this exception as:

@Retryable(value = {ItemStreamException.class}, maxAttempts = 5)
public void myMethod() {
   //..
}

Answer 2

You should use the spring-retry .

A RetryTemplate in combination with a TimeoutRetryPolicy will allow you do write the feature you want.

You need to add the @EnableRetry to the configuration class and @Retryable to the method.

Is the ItemReader needed to be instantiated first before the application can be started? If not you should just create the bean and have another method initialize it after the context refreshes.