CODEWITHSUNDEEP
javaxmlspringcsvspring-batch
Mohcine
Mohcine

Reputation: 57

File processing order with Spring Batch

I have a question about the order of processing of csv files with Spring Batch, I have several csv files to process, when I launch the spring batch I do not know in what order spring processes these files.

I would like to know how Spring chose the first file to treat ? and where can I find this setting ?

Is it possible to define a sort ? for example, processing files based on a date written on the file name? and how can I customize the choice of the first files to process?

Thank you all,

Upvotes: 3

Views: 2354

Answers (2)

KiranV
KiranV

Reputation: 1

@Bean
public MultiResourceItemReader<Employee> criminalBackgroundReader() {
    MultiResourceItemReader<Employee> resourceItemReader = new MultiResourceItemReader<Employee>();
    ClassLoader cl = this.getClass().getClassLoader();
    ResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(cl);

    Resource[] resources = null;
    try {
        resources = resolver.getResources("file:" + EmployeeInputFilePath);
    } catch (IOException e) {
        log.error("===== Error occurding reading input file directory: ", e);
        return null;
    }


    resourceItemReader.setResources(resources);
    resourceItemReader.setDelegate(flatFileRpReader());
    resourceItemReader.setComparator(new FileComparator());
    resourceItemReader.setStrict(false);
    return resourceItemReader;
}


public class FileComparator implements Comparator<Resource>{
    @Override
    public int compare(Resource file1, Resource file2) {
    //comparing based on File Name compare it to Enum-InputFileType
    return InputFileType.fromString(StringUtils.substringBefore(file1.getFilename(), ".")) .compareTo(InputFileType.fromString(StringUtils.substringBefore(file1.getFilename(),"."));
}
}

Upvotes: 0

Binu
Binu

Reputation: 784

Spring batch's MultiResourceItemReader uses Comparator<Resource> to preserve ordering. If we don't provide comparator, then the default ordering will be based on file name. If you want to give your custom sorting, you can write your own comparator logic like following(sort based on last modified time). 

public class FileModifiedComparator implements Comparator<FileSystemResource>{

        @Override
        public int compare(FileSystemResource file1, FileSystemResource file2) {
       //comparing based on last modified time
            return Long.compare(file1.lastModified(),file2.lastModified());
        }
 }

You can modify the comparator to check modify your sort logic such as file name, created etc. for eg: return file1.getFilename().compareTo(file2.getFilename()); or return Long.compare(file1.contentLength(),file2.contentLength()); and in the MultiResourceItemReader bean, set this comparator.

<bean id="fileModifiedComparator" class="FileModifiedComparator"/> 

<bean id="multiResourceReader"
    class=" org.springframework.batch.item.file.MultiResourceItemReader">
    <property name="resources" value="file:inputs/input*.csv" />
    <property name="delegate" ref="flatFileItemReader" />
    <property name="comparator" ref="fileModifiedComparator" />
</bean>

Upvotes: 3

Related Questions