CODEWITHSUNDEEP
springspring-bootspring-batch
JSF
JSF

Reputation: 344

Spring Batch - Chunk Oriented Processing without transactions

I've been researching various batch frameworks and I appreciate the abstraction and out-of-the-box features such as skip, retry, listeners etc. that Spring Batch brings with it and I'm familiar with Spring framework so this is a natural choice.

However, the batch flows I intent to create do not have transactional databases on either end of the reader-process-write flow. I desire to use Spring Batch to connect two systems through API's and still leverage the batch framework for tracking job executions and back the batch app with the Spring Batch database.

The remote API's support their own batching concepts, so, we can be sure to process 100 records and attempt to batch insert where the entire batch fails when one record is invalid. In that way, I would still like Spring Batch to "rollback" and retry each record individually.

Is it possible to leverage Spring Batch with its backing batch metadata database, to use skip and retry, without holding database connections or using transactions during chunk based processing?

Edit:

Based on Mahmoud's comment, I can use a DataSourceTransactionManager with the JobRepository and a ResourcelessTransactionManager with the chunk steps. So I will define a custom StepBuilderFactory:

@Component
public class MyStepBuilderFactory extends StepBuilderFactory {
  public MyStepBuilderFactory(JobRepository jobRepository) {
    super(jobRepository, new ResourcelessTransactionManager());
  }
}

Upvotes: 0

Views: 1302

Answers (1)

Mahmoud Ben Hassine
Mahmoud Ben Hassine

Reputation: 31600

A chunk-oriented tasklet requires a transaction manager to handle the "rollback" semantics you are looking for. If your step does not interact with a transactional resource, you can configure it to use a ResourcelessTransactionManager. This transaction manager is a NoOp implementation that can be used in such cases.

You can always use a DataSourceTransactionManager with the job repository to track job/step execution meta-data.

Upvotes: 1

Related Questions