I would like to create a spring batch project where batch does not use my datasource

Question

I have seen a lot of examples of Spring Batch projects where either (a) a dataSource is defined, or (b) no dataSource is defined.

However, in my project, I would like my business logic to have access to a dataSource, but I want Spring Batch to NOT use the dataSource. Is this possible?

This guy has a similar problem: Spring boot + spring batch without DataSource

Answer 1

Generally, using spring-batch without a database is not a good idea, since there could be concurrency issues depending on the kind of job you define. So at least an using an inmemory db is strongly advised, especially if you plan to use the job in production.

Using SpringBatch with SpringBoot will initialize an inmemory datasource, if you do not configure your own datasource(s).

Taking this into account, let me redefine your question as follows: Can my businesslogic use another datasource than springbatch is using to update its BATCH-tables? Yes, it can. As a matter of fact, you can use as many datasources as you want inside your SpringBatch Jobs. Just use by-name autowiring.

Here is how I do it: I always use Configuration class, which defines all the datasources I have to use in my Jobs

Configuration
public class DatasourceConfiguration {

    @Bean
    @ConditionalOnMissingBean(name = "dataSource")
    public DataSource dataSource() {
        // create datasource, that is used by springbatch
        // for instance, create an inmemory datasource using the 
        // EmbeddedDatabaseFactory
        return ...; 
    }

    @Bean
    @ConditionalOnMissingBean(name = "bl1datasource")
    public DataSource bl1datasource() {
        return ...; // your first datasource that is used in your businesslogic
    }

    @Bean
    @ConditionalOnMissingBean(name = "bl2datasource")
    public DataSource bl2datasource() {
        return ...; // your second datasource that is used in your businesslogic
    }
}

Three points to note:

SpringBatch is looking for a datasource with the name "dataSource", if you do not provide this EXACT (uppercase 'S') name as the name, spring batch will try to autowire by type and if it finds more than one instance of DataSource, it will throw an exception.

Put your datasource configuration in its own class. Do not put them in the same class as your jobdefinitions are. Spring needs to be able to instantiate the datasource-SpringBean with the name "dataSource" very early when it loads the context. Before it starts to instantiate your Job- and Step-Beans. Spring will not be able to do it correctly, if you put your datasource definitions in the same class as you have your job/step definitions.

Using @ConditionalOnMissingBean is not mandatory, but I found it a good practics. It makes it easy to change the datasources for unit/integration tests. Just provide an additional test configuration in the ContextConfiguration of your unit/IT test which, for instance, overwrites the "bl1Datasource" with an inMemoryDataSource:

Configuration
public class TestBL1DatasourceConfiguration {

    // overwritting bl1datasource with an inMemoryDatasource.
    @Bean
    public DataSource bl1datasource() {
        return new EmbeddedDatabaseFactory.getDatabase(); 
    }
}

In order to use the businesslogic datasources, use injection by name:

@Component
public class PrepareRe1Re2BezStepCreatorComponent {

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    private DataSource bl1datasource;

    @Autowired
    private DataSource bl2datasource;

    public Step createStep() throws Exception {
        SimpleStepBuilder<..., ...> builder =
                stepBuilderFactory.get("astep") //
                .<..., ...> chunk(100) //
                .reader(createReader(bl1datasource)) //
                .writer(createWriter(bl2datasource)); //

        return builder.build();
    }
}

Furthermore, you probably want to consider using XA-Datasources if you'd like to work with several datasources.

Edited:

Since it seems that you really don't want to use a datasource, you have to implement your own BatchConfigurer (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/core/configuration/annotation/BatchConfigurer.html) (as Michael Minella - the SpringBatch project lead - pointed out above).

You can use the code of org.springframework.batch.core.configuration.annotation.DefaultBatchConfigurer as a starting point for your own implementation. Simply remove all the datasource/transactionmanager code and keep the content of the if (datasource === null) part in the initialize method. This will initialize a MapBasedJobRepository and MapBasedJobExplorer. But again, this is NOT a useable solution in a productive environment, since it is not threadsafe.

Edited:

How to implement it:

Configuration class that defines the "businessDataSource":

@Configuration
public class DataSourceConfigurationSimple {
    DataSource embeddedDataSource;

    @Bean
    public DataSource myBusinessDataSource() {
        if (embeddedDataSource == null) {
            EmbeddedDatabaseFactory factory = new EmbeddedDatabaseFactory();
            embeddedDataSource = factory.getDatabase();
        }
        return embeddedDataSource;
    }
}

The implementation of a specific BatchConfigurer: (of course, the methods have to be implemented...)

public class MyBatchConfigurer implements BatchConfigurer {
    @Override
    public JobRepository getJobRepository() throws Exception {
        return null;
    }

    @Override
    public PlatformTransactionManager getTransactionManager() throws Exception {
        return null;
    }

    @Override
    public JobLauncher getJobLauncher() throws Exception {
        return null;
    }

    @Override
    public JobExplorer getJobExplorer() throws Exception {
        return null;
    }
}

And finally the main configuration and launch class:

@SpringBootApplication
@Configuration
@EnableBatchProcessing

// Importing MyBatchConfigurer will install your BatchConfigurer instead of
// SpringBatch default  configurer.
@Import({DataSourceConfigurationSimple.class, MyBatchConfigurer.class})

public class SimpleTestJob {

    @Autowired
    private JobBuilderFactory jobs;

    @Autowired
    private StepBuilderFactory steps;

    @Bean
    public Job job() throws Exception {
        SimpleJobBuilder standardJob = this.jobs.get(JOB_NAME)
                                                .start(step1());
        return standardJob.build();
    }

    protected Step step1() throws Exception {
        TaskletStepBuilder standardStep1 = this.steps.get("SimpleTest_step1_Step")
                                                     .tasklet(tasklet());
        return standardStep1.build();
    }

    protected Tasklet tasklet() {
        return (contribution, context) -> {
            System.out.println("tasklet called");
            return RepeatStatus.FINISHED;
        };
    }

    public static void main(String[] args) throws Exception {
        SpringApplication.run(SimpleTestJob.class, args);
    }
}