CODEWITHSUNDEEP
spring-batch
Manivannan
Manivannan

Reputation: 1

How to read the multi line reader using spring batch

I want to read the data from fixed format input data file like below, each record contains multiple lines and start with XYZ is the starting of the records and sub sequence lines are ABC & DEF also same records. again start with XYZ is the new records,

May i know which one is best using Spring batch ItemReader or ItemStream and any sample it will helpful. Advance Thanks

XYZ$19801234567S     2011234500001                                               
ABC$II9J234565443                                                                 
DEF$00095834753                                                                    
XYZ$198003453S       212345300003                                               
ABC$II43534503                                                                 
DEF$00035345303

Upvotes: 0

Views: 2085

Answers (1)

Victor Alcantara
Victor Alcantara

Reputation: 66

This is a common case when we have to handle multiple record types in a file. Look at the documentation here https://docs.spring.io/spring-batch/docs/4.2.x/reference/html/readersAndWriters.html#prefixMatchingLineMapper.

In your case you can define a mapper PatternMatchingCompositeLineMapper for your ItemReader which you have to define different tokenizers for each case.

Example:

@Bean
public PatternMatchingCompositeLineMapper patternMatchingLineMapper() {
    PatternMatchingCompositeLineMapper lineMapper = new PatternMatchingCompositeLineMapper();

   Map<String, LineTokenizer> tokenizers = new HashMap<>(3);
   tokenizers.put("XYZ*", xyzTokenizer());
   tokenizers.put("ABC*", abcTokenizer());
   tokenizers.put("DEF*", defTokenizer());

   lineMapper.setTokenizers(tokenizers);

   Map<String, FieldSetMapper> mappers = new HashMap<>(2);
   mappers.put("XYZ*", xyzFieldSetMapper());
   mappers.put("ABC*", abcFieldSetMapper());
   mappers.put("DEF*", defFieldSetMapper());

   lineMapper.setFieldSetMappers(mappers);

   return lineMapper;
}

Other solution you can use PatternMatchingCompositeLineTokenizer. In this case you can configure your line tokenizer as follow:

@Bean
public LineTokenizer patternCompositeLineTokenizer() throws Exception {
    FixedLengthTokenizer recordTypeXYZ = new FixedLengthTokenizer();
    ...

    FixedLengthTokenizer recordTypeABC = new FixedLengthTokenizer();
    ...

    FixedLengthTokenizer recordTypeDEF = new FixedLengthTokenizer();
    ...

    Map<String, LineTokenizer> tokenizers = new HashMap(3);
    tokenizers.put("XYZ*", recordTypeXYZ);
    tokenizers.put("ABC*", recordTypeABC);
    tokenizers.put("DEF*", recordTypeDEF);

    PatternMatchingCompositeLineTokenizer lineTokenizer = new PatternMatchingCompositeLineTokenizer();
    lineTokenizer.setTokenizers(tokenizers);

    return lineTokenizer;
}

@Bean
public FieldSetMapper<YourDomain> fieldSetMapper() { 
    return fieldSet -> {
        switch (fieldSet.readString("yourField")) {
            case XYZ: return new XyzDomain(...); 
            case ABC: return new AbcDomain(...);
            case DEF: return new DefDomain(...) 

            default: throw new IllegalArgumentException("Invalid record type ");
        } 
    };
}

@Bean
@StepScope
public FlatFileItemReader<YourDomain> itemReader() throws Exception {
    return new FlatFileItemReaderBuilder<YourDomain>()
            .name("itemReader")
            .resource(yourResource)
            .lineTokenizer(patternCompositeLineTokenizer())
            .fieldSetMapper(fieldSetMapper())
            .build();
}

Upvotes: 3

Related Questions