CODEWITHSUNDEEP
springspring-integrationfilteringspring-integration-sftp
capitaineballs
capitaineballs

Reputation: 165

Spring Integration File synchronizer : Accept files based on a pre defined list

I am transfering files from a remote to a local by Sftp, for processing. I want to only transfer .csv files, and I have a list of pre-defined filenames.

I couldn't find a FileListFilter that allows to specify multiple patterns and transfer if at least one if matched.

So far I have this code, that's woorking for ".csv" filtering.

The Integration Flow

@Bean
    public IntegrationFlow integFlow() {
        return IntegrationFlows
            .from(ftpMessageSource(), c -> poller())
            ... more processing

The MessageSource

public MessageSource<File> ftpMessageSource() {

            SftpInboundFileSynchronizer fileSynchronizer = new SftpInboundFileSynchronizer(sessionFactory);
            fileSynchronizer.setRemoteDirectory(remoteDirectory);
            fileSynchronizer.setDeleteRemoteFiles(true);
            fileSynchronizer.setFilter(new SftpRegexPatternFileListFilter(Constantes.EXTENSION));
            SftpInboundFileSynchronizingMessageSource ftpInboundFileSync = 
                new SftpInboundFileSynchronizingMessageSource(fileSynchronizer);

            ftpInboundFileSync.setLocalDirectory(new File(workDirectory));
            ftpInboundFileSync.setAutoCreateLocalDirectory(true);
            CompositeFileListFilter<File> compositeFileListFilter = new CompositeFileListFilter<>();
            compositeFileListFilter.addFilter(new RegexPatternFileListFilter(Constantes.EXTENSION));
            ftpInboundFileSync.setLocalFilter(compositeFileListFilter);
            return ftpInboundFileSync;
    }

Constantes.EXTENSION is a regex accepting .csv and .CSV. This works fine.

Say that I have a String list that contains "string1',"string2","string3" and I want to transfer every file of the form string1*, string2* or string3*. How would I proceed ?

Upvotes: 1

Views: 985

Answers (2)

Gary Russell
Gary Russell

Reputation: 174769

@SpringBootApplication
public class So59161698Application {

    public static void main(String[] args) {
        SpringApplication.run(So59161698Application.class, args);
    }

    private final String myPatterns = "foo,bar,baz";

    @Bean
    public FileListFilter<File> filter() {
        Set<String> patterns = StringUtils.commaDelimitedListToSet(this.myPatterns);
        return files -> Arrays.stream(files)
                .filter(file -> patterns.stream()
                        .filter(pattern -> file.getName().startsWith(pattern))
                        .findFirst()
                        .isPresent())
                .collect(Collectors.toList());
    }

    @Bean
    public ApplicationRunner runner(FileListFilter<File> filter) {
        return args -> {
            System.out.println(filter.filterFiles(new File[] {
                    new File("foo.csv"),
                    new File("bar.csv"),
                    new File("baz.csv"),
                    new File("qux.csv")
            }));
        };
    }

}

[foo.csv, bar.csv, baz.csv]

Upvotes: 1

Artem Bilan
Artem Bilan

Reputation: 121550

There is a CompositeFileListFilter:

* Simple {@link FileListFilter} that predicates its matches against <b>all</b> of the
* configured {@link FileListFilter}.

With the logic like:

public boolean accept(F file) {
    AtomicBoolean allAccept = new AtomicBoolean(true);
    // we can't use stream().allMatch() because we have to call all filters for this filter's contract
    this.fileFilters.forEach(f -> allAccept.compareAndSet(true, f.accept(file)));
    return allAccept.get();
}

So, you configure this CompositeFileListFilter with several SftpRegexPatternFileListFilter delegates and your files are going to be processed whenever they much at least one of the filters in the CompositeFileListFilter.

See more about filters in the docs: https://docs.spring.io/spring-integration/docs/current/reference/html/file.html#file-reading

Upvotes: 0

Related Questions