CODEWITHSUNDEEP
spring-batchspring-ioc
karthik
karthik

Reputation: 783

Spring Batch configure Multiple Readers/Processors/Writers for single JOB based on JobParameters

I have a spring batch application configured as per below link.

https://spring.io/guides/gs/batch-processing/

Now I have to configure step in such a way that it has to take FlatFileItemReader if user input is file and JdbcPagingItemReader if the input is SQL. I would pass the inputs(file/sql) as JobParameters.

From my understanding, everything in the above example by spring are singleton beans which are loaded into ApplicationContext when the application is started. As step can be configured only once with one Reader. How would I configure it to take different Reader's based on user input

I don't prefer creating multiple jobs if only the reader is changing.

I thought of using Factory/Strategy patterns but this are only achievable if Step is not a bean. Here all of them are beans which are loaded during application startup.

Regardless of patterns, a solution to use different Readers in step based on JobParameters would be helpful.

Upvotes: 1

Views: 8365

Answers (3)

Thomas Escolan
Thomas Escolan

Reputation: 1529

Remember that Spring configs (hence beans) can be conditioned to Spring profiles, environment variables, application context beans presence, properties, etc.

  • @Profile
  • @ConditionalOnBean @ConditionalOnMissingBean
  • @ConditionalOnClass @ConditionalOnMissingClass
  • @ConditionalOnExpression (SpringEL)
  • @ConditionalOnJava (JVM version)
  • @ConditionalOnJndi
  • @ConditionalOnProperty
  • @ConditionalOnResource
  • @ConditionalOnWebApplication @ConditionalOnNotWebApplication
  • @ConditionalOnEnabledEndpoint (see Actuator)

So you should be able to filter your factories (through @Configuration classes and @Bean methods as well), depending on the execution context.

Upvotes: 0

Niraj Sonawane
Niraj Sonawane

Reputation: 11075

You can implement this by using @StepScope. Put your bean name i.e FlatFileItemReader / or SQL in job parameters then use below code

sample implementation

java config way 

    @StepScope
        public Step step(@Value("#{jobParameters['beanId']}") final String beanId) {
        return stepBuilderFactory.get("step")
                .<POJO, POJO> chunk(10000)
                .reader(getReader(beanId))
                .processor(Processor)
                .writer(Writer)             
                .build();
     }


 getReader(String beanId)
     {
        return applicationContext.getBean(beanId);
     }

XML 

<batch:chunk reader="#{jobParameters['beanId']}" writer="writer" processor="processor" commit-interval="100" />

Upvotes: 2

dube
dube

Reputation: 5019

Your assumtion about the Beans is correct.

The fastest solution would be to just create two jobs, if you only have two flavors. But let's ignore reality and talk about theory:

You could create two steps, one for each flavor, and use a JobExecutionDecider to either do one or the other step (see docs).

Or you create your own ItemReader and let it create a delegate reader dynamically. If you use ItemStreamSupport or AbstractItemCountingItemStreamItemReader as base class, you get an open(ExecutionContext executionContext) method.

Sample code:

public class TicTacReader extends ItemStreamSupport implements ItemReader<TicTac>{

  protected ItemReader<TypedFieldSet<RecordType>> delegate;

  public TicTacReader() {
    super();
    setName(TicTacReader.class.getName());
  }

  @Override
  public void open(ExecutionContext executionContext) throws ItemStreamException {
    super.open(executionContext);
    // TODO: use the appropriate reader
    this.delegate = new VoodooReader();
  }

      @Override
  public TicTac read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
    TypedFieldSet<RecordType> line = this.delegate.read();
    // TODO ...
 }
}

Upvotes: 4

Related Questions