CODEWITHSUNDEEP
spring-bootspring-data-jpaspring-batch
G V
G V

Reputation: 85

Spring batch file resume after Server failure

I have configured Spring Boot Batch to process Fixed length flat file. I read and split columns by using FlatFileItemReader, FixedLengthTokenizer and Writing data into Database by using ItemWriter, JPA Repository.

I have a scenario like, My Server was crashed or it was stopped at the time of file processing. At this point half of the file was processed(means half of the data wrote into DB). When it comes to next Job(when server was running up) the file has to start from where it stops.

For Example, A file having 1000 lines, Server was shutdown after processing 500 rows. In the next Job, The file has to start from 501 row.

I googled for solution but nothing relevant. Any help appreciated.

Upvotes: 5

Views: 1075

Answers (1)

Sabir Khan
Sabir Khan

Reputation: 10142

As far as I know, what you are asking ( restart at chunk level ) doesn't automatically exist in Spring Batch API & is something that programmer has to implement on his/her own.

Spring Batch provides Job restart feature via JobOperator.restart . This is a job level restart and a new execution id will be created for next run & whole of the job will rerun as there are other concerns like somebody put in a new file or renamed existing file to process in place of old file , how batch will know that its same input file content wise or db not changed since last run?

Due to these concerns , its imperative that programmer handles these situations via custom code.

Second concern is that when there is a server failure, job status would still be STARTED & not FAILED since it happens all of a sudden and framework couldn't update status correctly.

Following steps you need to implement ,

1.Implement a custom logic to decide if last job execution was successful or restart is needed.

2.If restart is needed, mark previous job execution as FAILED & then use JobOperator.restart(long executionId) - For a non - partitioned job , only useful impact would be the marking of job status to be correct as FAILED but whole job will restart from beginning.

There are many scenarios like,

a)job status is STARTED but all steps are marked COMPELTEDetc

b)For a partitioned job, few steps are completed , few failed & few in started etc

3.If restart is not needed, launch a new job using - JobLauncher.run.

So with above steps, you see that a real chunk level job restart is not achieved but above steps are primary things that you first need to understand & implement.

Next would be to changing your input at job restart i.e. you devise a mechanism to mark input records as processed for processed chunks ( i.e. read , processed & written ) & have a way to know what input records are not processed - then in next job run you feed modified input that is still unprocessed. So its all going to be your use case specific custom logic.

I am not aware of any inbuilt mechanism in the framework itself to achieve this. To me a Job Restart is a brand new job execution with modified / reduced input.

Upvotes: 3

Related Questions