CODEWITHSUNDEEP
springcsvspring-batchinputstreamreader
gstackoverflow
gstackoverflow

Reputation: 37034

How to read csv with unknow column names and with unknow column count using in spring batch?

I have following the FlatFileItemReader configuration for my step:

@Bean
@StepScope
public FlatFileItemReader<RawInput> reader(FieldSetMapper<RawInput> fieldSetMapper, @Value("#{jobParameters['files.location']}") Resource resource) {
    var reader = new FlatFileItemReader<RawInput>();
    reader.setName("my-reader");
    reader.setResource(resource);
    var mapper = new DefaultLineMapper<RawInput>();
    mapper.setLineTokenizer(crmCsvLineTokenizer());
    mapper.setFieldSetMapper(fieldSetMapper);
    mapper.afterPropertiesSet();
    reader.setLineMapper(mapper);
    return reader;
}

RawInput contains 1 field so it allows me to read csv with single column. For now requirements were changes and now I have to be able to read any csv file with any amount of rows thus instead of RawInput I need to pass array somehow. is it possible with FlatFileItemReader or maybe I should change implementation ?

Upvotes: 0

Views: 1055

Answers (1)

gstackoverflow
gstackoverflow

Reputation: 37034

It works:

    var reader = new FlatFileItemReader<List<String>>();
    reader.setName("reader");
    reader.setResource(resource);
    //line mapper
    var lineMapper = new DefaultLineMapper<List<String>>();        
    lineMapper.setLineTokenizer(new DelimitedLineTokenizer());
    lineMapper.setFieldSetMapper(myFieldSetMapper); // see implementation below
    lineMapper.afterPropertiesSet();

    reader.setLineMapper(lineMapper);
    return reader;


@Component
public class MyFieldSetMapper implements FieldSetMapper<List<String>> {

    @NonNull
    @Override
    public List<String> mapFieldSet(@NonNull FieldSet fieldSet) {
        return Arrays.stream(fieldSet.getValues())
                .map(StringUtils::lowerCase) // optional
                .map(StringUtils::trimToNull) // optional
                .collect(Collectors.toList());
    }
}

Upvotes: 2

Related Questions