CODEWITHSUNDEEP
javaspringspring-bootspring-batchspring-retry
Sam Fisher
Sam Fisher

Reputation: 63

How to use Spring Batch to connect to server?

I need help from experienced Spring Batch programmers for a particular problem I am having.

I am new to Spring Batch. It has detailed documentation for each class or interface, but next to nothing in all of it working together.

The Scenario:
I have a database hosted on the cloud, So I have to make REST calls to get some data from it and save related data to it.
- Data retrieval will return a JSON response with data I queried.
- Data save will return a JSON response on how many rows were added etc.
- All responses will have a valid HTTP status code.
- A transaction is complete when the save call is successful with an Http code of 200 and data which shows how many records were inserted is received.

The connection may not always be available, In that case the program must keep retrying every 5 minutes until the whole task is complete.

What I chose not to do
I could do some dirty Java tricks (which were surprisingly recommended by many in stack overflow)

  1. Threads and sleep (Too crude)
  2. Spring's @Scheduled (The scheduler keeps running even after job completion)

What I tried
So I decided to use Spring Batch since it seemed to be a framework made for this.

  1. I have no file tasks, so I used a Tasklet instead of Readers and Writers.

  2. The Tasklet interface can return only FINISHED status code. No codes for FAILURE

  3. So, inside the tasklet, I set a custom value in the StepContext and retrieved my custom value in a StepExecutionListener and accordingly configured ExistStatus of the Step to FAILURE

  4. To handle this workaround I had to configure a JobExecutionListener to make the Job fail accordingly.

Apart from all these above work-arounds,

  1. Spring batch does not have any scheduling. I have to end up using another scheduler.
  2. Spring Batch's retry within a step is valid only for ItemReader,ItemWriter etc and not for tasklets

The Question

  1. Is Spring Batch right for this situation?

  2. Is my design correct? It seems very "hack"-ey.

I need help with the most efficient way to handle my scenario

Upvotes: 1

Views: 644

Answers (1)

Igor Konoplyanko
Igor Konoplyanko

Reputation: 9374

I was using spring batch for similar case - as a execution engine to process large files which resulted as lots of REST requests to other systems.

What Spring batch brought for me:

  • execution engine/model for large dependent operations. In other words I could maintain my input as one single entry point and have 'huge' transaction on top of other small operations.
  • Possibility to see execution results and monitor them.
  • Retriability of batch operations. This is one of the best thing in spring batch
  • it allows you design your operation in such manner that if something goes wrong in the middle of execution, you can simply restart it and continue from failing point. But you need to invest some effort to maintain this.

More on business cases here: https://docs.spring.io/spring-batch/trunk/reference/html/spring-batch-intro.html#springBatchUsageScenarios

So you need to check carefully those business cases and answer yourself if you really need them. So far what you have described - I really don't see benefit of spring batch for you.

Upvotes: 1

Related Questions