How to pass data between Spring Batch jobs

Question

I'm familiar with how to pass data between steps in a Spring Batch job. But what happens when you job is composed of many smaller jobs? In the example below, I would like to set some data in the JobExecutionContext at the end of the first job, siNotificationJob. Then, that data could be read from the JobExecutionContext of StepExecutionContext in the next job, ciNotificationJob. Do I need to promote this data somehow? I can't seem to see the results in the Job Parameter Extractor defined in step 'ciNotificationJob' that I use to configure my job parameters.

Thoughts?

Andrew

anschoewe · Accepted Answer

I was able to resolve this. I'll show you through example how I solved it. It was complicated but I think the end result is fairly easy to understand.

I have one overall job called 'notificationJob'. It has three steps that calls 3 different jobs (not steps). Each of these jobs can run independently, or be called from within the top level 'notificationJob'. Also, each sub-job has many steps. I'm not going to show all those steps here, but just wanted to highlight that these are complete jobs themselves with more multliple steps.

The key is being able to extract the results from one job and read them in the next job. Now, you could do this multiple ways. One way would be to output the result from one job into a DB or text file and then have the next job read from that file/table. Since I wasn't dealing with that much data, I passed the information around in memory. So, you'll notice the job-parameter-extractors. You can either rely on a built-in implementation of a paramter extractor, or you can implement your own. I actually use both. All they do is extract the value from the StepExecution and then we'll need to promote/move them to the next sub-job.

    
    
        
            OUTPUT
        
    



    
    
        
                outputName
                FUTURE_SI_JOB_RESULTS
            
    
 


    
    
        
                outputName
                FUTURE_CI_JOB_RESULTS

Finally, you'll notice that there is a parent job listener. This is the magic that transfer the state from one job and makes it available to the next. Here is my implementation of the class that does that.

    
    
 




package jobs.permnotification;

import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;

public class ParentJobListener implements JobExecutionListener
{

    private JobExecution parentExecution;

    @Override
    public void beforeJob(JobExecution jobExecution)
    {
        this.parentExecution = jobExecution;
    }

    @Override
    public void afterJob(JobExecution jobExecution)
    {
        // TODO Auto-generated method stub

    }

    public void setParentExecution(JobExecution parentExecution)
    {
        this.parentExecution = parentExecution;
    }

    public JobExecution getParentExecution()
    {
    return parentExecution;
    }

}


package jobs.permnotification;

import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;

public class JobStateListener implements JobExecutionListener
{
    private ParentJobListener parentJobListener;


    @Override
    public void beforeJob(JobExecution jobExecution)
    {
        if(parentJobListener == null || parentJobListener.getParentExecution() == null) return;
        passStateFromParentToJob(StepKey.FUTURE_SI_JOB_RESULTS.toString(), jobExecution);
        passStateFromParentToJob(StepKey.FUTURE_CI_JOB_RESULTS.toString(), jobExecution);
        passStateFromParentToJob(StepKey.OUTPUT.toString(), jobExecution);
    }

    @Override
    public void afterJob(JobExecution jobExecution)
    {
        if(parentJobListener == null || parentJobListener.getParentExecution() == null) return;
        //take state from child step and move it into the parent execution context
        passStateFromJobToParent(StepKey.FUTURE_SI_JOB_RESULTS.toString(), jobExecution);
        passStateFromJobToParent(StepKey.FUTURE_CI_JOB_RESULTS.toString(), jobExecution);
        passStateFromJobToParent(StepKey.OUTPUT.toString(), jobExecution);
    }

    private void passStateFromJobToParent(String key, JobExecution jobExecution)
    {
        Object obj = jobExecution.getExecutionContext().get(key);
        if(obj != null)
            parentJobListener.getParentExecution().getExecutionContext().put(key, obj);
    }

    private void passStateFromParentToJob(String key, JobExecution jobExecution)
    {
        Object obj = parentJobListener.getParentExecution().getExecutionContext().get(key);
        if(obj != null)
            jobExecution.getExecutionContext().put(key, obj);
    }

    public void setParentJobListener(ParentJobListener parentJobListener)
    {
        this.parentJobListener = parentJobListener;
    }

    public ParentJobListener getParentJobListener()
    {
        return parentJobListener;
    }

}

How to pass data between Spring Batch jobs

Answers (2)

Related Questions