CODEWITHSUNDEEP
spring-batch
explorer
explorer

Reputation: 516

Reading from streams instead of files in spring batch itemReader

I am getting a csv file as a webservice call which needs to be laoded. Right now I am saving it in temp directory to provide it as setResource to Reader.

Is there a way to provide stream(byte[]) as is instead of saving the file first?

Upvotes: 9

Views: 16861

Answers (2)

Thrax
Thrax

Reputation: 1964

The method setResource of the ItemReader takes a org.springframework.core.io.Resource as a parameter. This class has a few out-of-the-box implementations, among which you can find org.springframework.core.io.InputStreamResource. This class' constructor takes a java.io.InputStream which can be implemented by java.io.ByteArrayInputStream.

So technically, yes you can consume a byte[] parameter in an ItemReader.

Now, for how to actually do that, here are a few ideas :

  1. Create your own FlatFileItemReader (since CSV is a flat file) and make it implement StepExecutionListener

    public class CustomFlatFileItemReader extends FlatFileItemReader implements StepExecutionListener { }

  2. Override the beforeStep method, do your webservice call within and save the result in a variable

    private byte[] stream;

    @Override public void beforeStep(StepExecution stepExecution) {

      // your webservice logic
  stream = yourWebservice.results();

    }

  3. Override the setResource method to pass this stream as the actual resource.

    @Override public void setResource(Resource resource) {

     // Convert byte array to input stream
 InputStream is = new ByteArrayInputStream(stream);

 // Create springbatch input stream resource
 InputStreamResource res = new InputStreamResource(is);

 // Set resource
 super.setResource(res);

    }

Also, if you don't want to call your webservice within the ItemReader, you can simply store the byte array in the JobExecutionContext and get it in the beforeStep method with stepExecution.getJobExecution().getExecutionContext().get("key");

Upvotes: 9

Guilherme
Guilherme

Reputation: 897

I am doing right now with FlaFileItemReader, reading a file from Google Storage. No needed to extends:

    @Bean
@StepScope
public FlatFileItemReader<MyDTO> itemReader(@Value("#{jobParameters['filename']}") String filename) {
    
    InputStream stream = googleStorageService.getInputStream(GoogleStorage.UPLOADS, filename);
    
    return new FlatFileItemReaderBuilder<MyDTO>()
            .name("myItemReader")
            .resource(new InputStreamResource(stream)) //InputStream here
            .delimited()
            .names(FIELDS)
            .lineMapper(lineMapper()) // Here is mapped like a normal File
            .fieldSetMapper(new BeanWrapperFieldSetMapper<MyDTO>() {{
                setTargetType(MyDTO.class);
            }})
            .build();
}

Upvotes: 0

Related Questions