CODEWITHSUNDEEP
spring-batch
Vishal Singh
Vishal Singh

Reputation: 45

Reading file dynamically in spring-batch

I am trying to transfer any files (video,txt etc) between different endpoints (pc, s3, dropbox, google drive) using spring-batch on a network. For that, I am getting json file containing list of files location(url) to be transferred (assume I can access those location).

So, how do I tell the reader to read the input once my controller is hit (in which job is created) and not at the time of starting spring-boot application?

I have tried adding "spring.batch.job.enabled=false" which stops spring-batch to start automatically but my concern is where should I write setting my resource line that will be provided to ItemReader :

FlatFileItemReader<String> reader = new FlatFileItemReader<String>();
reader.setResource(someResource);

Because during setting resources I am getting NullPointerException.

Upvotes: 0

Views: 2311

Answers (1)

Mahmoud Ben Hassine
Mahmoud Ben Hassine

Reputation: 31730

The Running Jobs from within a Web Container explains that with a code example. Here is an except:

@Controller
public class JobLauncherController {

   @Autowired
   JobLauncher jobLauncher;

   @Autowired
   Job job;

   @RequestMapping("/jobLauncher.html")
   public void handle() throws Exception{
    jobLauncher.run(job, new JobParameters());
   }
}

In your case, you need to extract the file name from the request and pass it as a job parameter, something like:

@RequestMapping("/jobLauncher.html")
public void handle() throws Exception{
   URL url = // extract url from request
   JobParameters parameters = new JobParametersBuilder()
        .addString("url", url)
        .toJobParameters();
   jobLauncher.run(job, parameters);
}

Then make your reader step-scoped and dynamically extract the file from job parameters:

@StepScope
@Bean
public FlatFileItemReader flatFileItemReader(@Value("#{jobParameters['url']}") URL url) {
   return new FlatFileItemReaderBuilder<String>()
        .resource(new UrlResource(url))
        // set other properties
        .build();
}

This is explained in the Late Binding of Job and Step Attributes section.

Upvotes: 1

Related Questions