CODEWITHSUNDEEP
javaspringspring-batchbatch-processing
Bug
Bug

Reputation: 483

how to read both comma separated and pipe line separated csv file in a single item reader in spring batch

I am new to sprig batch. I have a folder which contain multiple csv file, I have implemented MultiResourceItemReader () to read those file . It is working only if all csv file are pipe line ("|") separated.

I want to read both comma (",") separated csv and pipe line separated csv using single reader. Is it possible ? if yes how ?

Here is my code 

@Bean
@StepScope
public MultiResourceItemReader<Person> multiResourceItemReader(@Value("#{jobParameters[x]}") String x,@Value("#{jobParameters[y]}") String y,@Value("#{jobParameters[z]}") String z) {

    Resource[] resourcessss = null;
    ClassLoader cl = this.getClass().getClassLoader();
    ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(cl);
    try {
        resourcessss = resolver.getResources("file:" + z);
    }catch(Exception e) {

    }
    MultiResourceItemReader<Person> resourceItemReader = new MultiResourceItemReader<Person>();
    resourceItemReader.setResources(resourcessss);
    resourceItemReader.setDelegate(reader());

    return resourceItemReader;
}



@Bean
public FlatFileItemReader<Person> reader() {
    FlatFileItemReader<Person> reader = new FlatFileItemReader<Person>();
    reader.setLineMapper(new DefaultLineMapper() {
        {
            setLineTokenizer(new DelimitedLineTokenizer() {
                {
                    setNames(new String[]{"Id","postCode"});
                }
                {
                    setDelimiter("|");
                }
            });
            setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {
                {
                    setTargetType(Person.class);
                }
            });
        }
    });
    return reader;
}

Upvotes: 0

Views: 1323

Answers (1)

Michael Minella
Michael Minella

Reputation: 21493

Take a look at the PatternMatchingCompositeLineTokenizer. There, you can use a Pattern to identify what records get parsed by what LineTokenizer. In your case, you'd have one Pattern that identifies comma delimited records and map them to the tokenizer that parses via commas. You'd also have a Pattern that identifies records delimited by pipes and maps those to the appropriate LineTokenizer. It would look something like this:

    @Bean
    public LineTokenizer compositeLineTokenizer() throws Exception {
        DelimitedLineTokenizer commaTokenizer = new DelimitedLineTokenizer();

        commaTokenizer.setNames("a", "b", "c");
        commaTokenizer.setDelimiter(",");
        commaTokenizer.afterPropertiesSet();

        DelimitedLineTokenizer pipeTokenizer = new DelimitedLineTokenizer();

        pipeTokenizer.setNames("a", "b", "c");
        pipeTokenizer.setDelimiter("|");

        pipeTokenizer.afterPropertiesSet();

        // I have not tested the patterns here so they may need to be adjusted
        Map<String, LineTokenizer> tokenizers = new HashMap<>(2);
        tokenizers.put("*,*", commaTokenizer);
        tokenizers.put("*|*", pipeTokenizer);

        PatternMatchingCompositeLineTokenizer lineTokenizer = new PatternMatchingCompositeLineTokenizer();

        lineTokenizer.setTokenizers(tokenizers);

        return lineTokenizer;
    }

Upvotes: 1

Related Questions