Multiple Spring Batch jobs in one Grails application

Question

I have a Grails application that uses the SpringBatch plugin to control batch loading of files into a database. I've got two batch configs in one Grails application, let's call them OneBatchConfig.groovy and TwoBatchConfig.groovy. They do very similar things, just with slightly different processing. For example:

OneBatchConfig.groovy

beans {
    xmlns batch:"http://www.springframework.org/schema/batch"

    batch.job(id: "batchJob1") {
        batch.step(id: 'loadFile') {
            batch.tasklet {
                batch.chunk(
                    reader:    'fileReader',
                    processor: 'compositeProcessor',
                    writer:    'dbWriter')
                }
            }
        }

        // Ignore the fileReader and dbWriter for now

        compositeProcessor(CompositeItemProcessor) {
            delegates = [
                ref('filterInvalidNames'),
                ref('filterInvalidDepartments')
            ]
        }

        // Ignore the details of the filter* processors
    }
}

TwoBatchConfig.groovy

beans {
    xmlns batch:"http://www.springframework.org/schema/batch"

    batch.job(id: "batchJob2") {
        batch.step(id: 'loadFile') {
            batch.tasklet {
                batch.chunk(
                    reader:    'fileReader',
                    processor: 'compositeProcessor',
                    writer:    'dbWriter')
                }
            }
        }

        // Ignore the fileReader and dbWriter for now

        compositeProcessor(CompositeItemProcessor) {
            delegates = [
                ref('filterInvalidNames'),
                ref('filterInvalidCities')
            ]
        }

        // Ignore the details of the filter* processors
    }
}

Both batch jobs read in a file, perform some processing, and save the results to the database. The configuration of the fileReader is different for each batch job, in that they are reading different files, for instance, but what's more obvious above is that the processing applied inside each job is different. The first batch job filters out some records based on names and departments, while the second job filters out some records based on names and cities.

However, those filtering steps are configurable, in that the first job might be filtering out records with name = 'John', while the second job might be filtering out records with name = 'Steve'. That configuration is inside the batch job definition itself, not in the code.

So my question is this: in my testing, it appears as though all of the beans that are defined in both of these *BatchConfig.groovy files are part of a global namespace; including the names of the steps. This means that having a step called loadFile in both OneBatchConfig.groovy and TwoBatchConfig.groovy is a bad idea, and only one of those steps will really exist after the Grails application starts up. This also means that the filterInvalidNames bean can only exist once, and therefore the same filtering will be applied to both jobs (somewhat invisibly).

First, is that understanding correct? It makes some sense, given how other beans, like dataSource, are just available without having to do much of anything.

But if so, it also creates some havoc when managing a large number of *BatchConfig.groovy files within a single Grails application. Is there a solution other than requiring the use of unique names for every single step and every single bean across all *BatchConfig.groovy files that are created? Like the ability to apply a namespace of some kind to the bean names within each batch.job block?

Maybe I'm just missing something simple in how to configure each job so it exists separately from other jobs?

Answer 1

The components in a job are simply spring beans that are loaded into the global Grails (Spring) context. This means You'll have to give your components globally unique names. I simply prefix each step with the job name, though this can make for some long bean names.

If you were using spring-batch directly, you could use the AutomaticJobRegistrar to get around this. The Grails plugin does not currently do this. I've commented on your issue on github to reflect this request. https://github.com/johnrengelman/grails-spring-batch/issues/23