JSR 352 Spring Batch: Transaction Management

Question

I try to use Spring Batch in JSR 352 mode in Websphere. (Websphere 8.0, Spring Batch 3.0.1)

As far as I understand the documentation, spring should handle the transactions, i.e. begin a transaction before calling the ItemReader of a step, committing the transactions after calling the ItemWriter, etc.

However, in my case no transaction is active when the ItemReader is called (userTransaction.getStatus() == 6). My code works if I start the transaction myself in the itemReader, but my understanding is that I shouldn't have to do that.

I suspect the problem is in the way I set up my batch.

This is a sample code that shows the problem:

META_INF/batch.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:tx="http://www.springframework.org/schema/tx"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd">


<tx:jta-transaction-manager />

</beans>

META-INF/batch-jobs/samplebatch3.xml:

<?xml version="1.0" encoding="UTF-8"?>


<job version="1.0"
     id="samplebatch3" 
     xmlns="http://xmlns.jcp.org/xml/ns/javaee" 
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/jobXML_1_0.xsd">
 <step id="step1">
     <chunk  checkpoint-policy="item"
            item-count="5">
            <reader ref="my.jbatchtest.samplebatch3.SampleReader" />
            <processor ref="my.jbatchtest.samplebatch3.SampleProcessor"/>
            <writer ref="my.jbatchtest.samplebatch3.SampleWriter" />
     </chunk> 
  </step>
</job>

The ItemReader:

package my.jbatchtest.samplebatch3;

import java.io.Serializable;

import javax.batch.api.chunk.ItemReader;
import javax.naming.InitialContext;
import javax.transaction.UserTransaction;

import org.xadisk.connector.outbound.XADiskConnectionFactory;

public class SampleReader implements ItemReader {

    private UserTransaction utx;


    public SampleReader() {
        // TODO Auto-generated constructor stub
    }

    @Override
    public void open(Serializable checkpoint) throws Exception {

        utx = (UserTransaction) new InitialContext().lookup("jta/usertransaction");   
        System.out.println("Status before begin:"+utx.getStatus());
        utx.begin();
        System.out.println("Status after begin:"+utx.getStatus());

    }

    @Override
    public void close() throws Exception {
        // TODO Auto-generated method stub

    }

    @Override
    public Object readItem() throws Exception {
        // TODO Auto-generated method stub
        return null;
    }

    @Override
    public Serializable checkpointInfo() throws Exception {
        // TODO Auto-generated method stub
        return null;
    }

}

The debugging output from the ItemReader:

[07.10.14 12:52:48:881 CEST] 00000039 SystemOut     O Status before begin:6
[07.10.14 12:52:48:881 CEST] 00000039 SystemOut     O Status after begin:0

My questions are:

1. is my understanding correct, that spring batch should manage the transactions?
2. then why doesn't it do so?

Answer 1

Let me chime in on a few of the issues brought up here. Full disclosure, I'm currently the project lead for Spring Batch and I was also on the expert group for JSR-352.

What is the status of Spring Batch and JSR-352
Spring Batch as of release 3.0.0 is JSR-352 compliant for all SE requirements. JSR-352's requirements are broken up into SE and EE groupings. We have not addressed any of the EE features as specified in the spec that differ from Spring Batch's "normal" processing.

Why hasn't Spring Batch implemented the EE specific features?
We went down the route of only implementing SE requirements for one simple reason. Those are the only requirements we can verify are compliant with the spec through the available TCK. The only way to validate the EE specific requirements is to run the TCK as part of the certification of an application server (Oracle's CTS), which Spring Batch obviously is not a part of, nor has access to (given that it must be licensed).

We have requested a version of the TCK that we can validate the EE features against and as of today, the response we have received has been "In opening up the TCK for public contributions on GitHub, this is certainly an enhancement we'd be happy to accept help with."

Like any good developers, we do not want to develop new features without tests. We want to be able to validate what we have implemented meets the requirements. In this case, that validation step lies in executing the TCK against our code, something we can't do at this time. Not being able to execute the TCK puts us at risk of implementing the EE features in the way we "think" is correct, releasing them, and then having to change their behavior later if/when the TCK can be run which is in direct contradiction to the portability that the JCP is trying to provide.

What about BATCH-2240?
BATCH-2240 as it currently stands is not a request to fix how the Spring Batch JSR-352 implementation behaves. On the contrary, it requests that the behavior of wrapping the ItemStream#open() and ItemStream#close() methods in transactions which would impact normal Spring Batch processing only. Fixing BATCH-2240 would not directly address how Spring Batch's JSR-352 behaves. It is marked as a minor improvement because it isn't viewed as a bug in our JSR-352 implementation, it's viewed as an additional feature for our regular Spring Batch processing.

To answer the original questions
1. Is my understanding correct, that spring batch should manage the transactions? - Yes, your understanding is correct. Spring Batch does handle transactions. With a chunk based step, the open and close methods are called outside of a transaction allowing state of the implementing component to be reset. Within Spring Batch, there is nothing stopping you from wrapping your code in a transaction using the TransactionTemplate to get similar functionality.
2. Then why doesn't it do so? - I hope the above points address this question.

Conclusion
We want to address the EE specific features of JSR-352 once we have a way to validate that we've addressed them in a way that guarantees the portability that the JCP is attempting to accomplish. We believe the right way to do that is to have a standardized TCK that we can run. Once that is addressed, addressing the differences between the EE version of the spec and Spring Batch will be a high priority.

Answer 2

On your samplebatch3.xml, you need to declare a tasklet between your step and chunk tags, and on the tasklet tag, you put a reference to the transaction manager Spring Batch will use, otherwise he wont know which transaction manager to use. See section 5.1.1 on this page:

http://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html

Edited: sorry, I forgot you use jsr-352 instead of plain Spring batch, see the link below, it specifies some settings you need to pass to the framework, one of them is the transaction manager

http://docs.spring.io/spring-batch/trunk/reference/html/jsr-352.html

And on this article, there´s a example of configuring:

https://blog.codecentric.de/en/2014/08/writing-jsr-352-style-jobs-spring-batch-part-1-configuration-options/

See section "How to customize the standard configuration"

Answer 3

After reading and stepping through the spring batch code, and reading the JSR 352 specs, I think this might be a bug in spring batch.

I don't like blaming problems on well-tested libraries like spring, so I still may be wrong. Here is what I found:

Spring calls the open() method of the reader and writer, and then executes the main batch cycle of reading, processing and writing. After completing the cycle, close() is called on the reader and writer.

The problem is, that spring batch only runs the main batch cycle inside a transaction context. It doesn't start a transaction for the open() and close() calls.

According to the specs these calls should be run in their own transaction. This is from the JSR 352 specs:

11.6 Regular Chunk Processing

1. <Create StepContext>
2. <Store step level properties in StepContext>
3. <->[StepListener.beforeStep...] // thread A
4. [<begin transaction> ]
5. <->ItemReader.open // thread A
6. <->ItemWriter.open // thread A
7. [<commit transaction> ]
8. // chunk processing:
9. <repeat until no more items> {
   a. <begin checkpoint [<begin transaction> ]>
   b. <repeat until commit criteria reached> {
        i. <->ItemReader.readItem // thread A
       ii. <->ItemProcessor.processItem // thread A
      iii. <add item to buffer>
   c. }
   d. <->ItemWriter.writeItems // thread A
   e. <->[ItemReader.checkpointInfo] // thread A
   f. <->[ItemWriter.checkpointInfo] // thread A
   g. <Store StepContext persistent area>
   h.
   i. <commit checkpoint (commit transaction)>
10. }
11. [<begin transaction> ]
12. <->ItemWriter.close // thread A
13. <->ItemReader.close // thread A
14. [<commit transaction> ]
15. <->[StepListener.afterStep...] // thread A
16. <Store StepContext persistent area>
17. <Destroy StepContext>

Line 4, 7, 11 and 14 never happen in spring batch.

I see this behaviour in my code (no open transaction in open() and close()), and it is also reflected in the spring batch code: obvious transaction bracket for main cycle, no transaction code for open() and close())

I found no way to post a bug on the spring.io site, just a link to stackoverflow.com. Maybe someone from the spring team will see this and give some feedback (or at least pass this on to the developers)

Upate: This seems to be a known issue: https://jira.spring.io/browse/BATCH-2240 The notation of the specs seem to indicate that starting a transaction here is optional, which makes coding the reader and writer a hassle, since I have to check whether I have a valid transaction, and create my own transaction bracket is necessary.