CODEWITHSUNDEEP
springspring-batch
Brian
Brian

Reputation: 227

Accessing StepExecutionContext from Item Writer in Remote Chunk Processor

I've implemented a simple Spring Batch solution that reads data from an XML file and writes it to a database. The solution works well but due to scalability requirements I'm converting this simple implementation to a more scalable implementation that uses Remote Chunking. I've run into an issue attempting to access the JobExecutionContext from the remote ItemWriter. My ItemWriter is shown below (non essentials removed for brevity) and works fine in the non remote chunking implementation.

public Object myObject;

public void write(List<? extends Employee> items) throws Exception
{   
    // do stuff
// use myObject here
}

public void beforeStep(StepExecution stepExecution_p)
{
     myObject = stepExecution_p.getJobExecution().getExecutionContext().get("myObject");            
}

However when I run the remote chunking implementation the beforeStep method above isn't invoked and so I don't have a handle on myObject when the write method is later invoked. I understand that the before method is not being invoked before the step begins because the ItemReader and ItemWriter are distributed and running in separate JVMs, therefore there is no way for the beforeStep to be invoked on the remote ItemWriter.

I need to find a way of getting a handle on the steExecutionContext in my remote ItemWriter so that I can get myObject from that context. Is there a way in which I can get a handle on the stepExecutionContext in the remote Item Writer or any other way of passing useful data into the remote ItemWriter? I looked at late binding to the remote ItemWriter but apparently step scoping of remote components (ItemWriter in this case) is not supported.

<bean id="itemWriterSlave" class="com.....ItemWriterSlave" scope="step"> 
    <property name="myObject" value="#{stepExecutionContext[myObject]}" />
</bean>

Does anyone know how I can access data passed from previous step (via the stepExecutionContext) in a remote ItemWriter? Any suggestions would be greatly appreciated.

Upvotes: 2

Views: 1581

Answers (1)

Michael Minella
Michael Minella

Reputation: 21463

Spring Batch provides two different ways to do multi-JVM scaling: remote partitioning and remote chunking. Remote Partitioning, while the slave steps are not part of the job that contains the master step, they are still true steps that have all the rights and responsibilities of a Spring Batch step (access to the job repository, slaves get StepExecutions, etc).

However, remote chunking is different. The slaves are not actually true steps. They are really just remotely deployed components used by the master. Because of that, they don't have StepExecutions or related ExecutionContexts. The best way to pass data from previous steps to slaves in remote chunking is to persist it somewhere outside of the job.

Upvotes: 3

Related Questions