CODEWITHSUNDEEP
javaspringspring-integrationspring-dsl
kacpr
kacpr

Reputation: 392

AcceptOnceFileListFilter and renaming a file in Spring Integration

I am writing an IntegrationFlow for SFTP, to copy only the latest file in remote directory, to local. I also want to avoid copying the same file twice so I added an AcceptOnceFileListFilter. Unfortunately the remote files come in form of IGNORED_STRING_1008202211:22:33.csv, as you see there is a date-time stamp, based on which I determine the latest file, but the colon symbol is not accepted for filenames in Windows, so I had to apply a function to rename it. Now this messes with the AcceptOnceFileListFilter, I've debugged and the original file name is checked once, then it is compared to the modified one (without colons) - so it always gets downloaded again.

Sftp.inboundAdapter(sftpSf)
    .filter(new ChainFileListFilter<>() {{
        addFilter(new LatestFileOnly());
        addFilter(new AcceptOnceFileListFilter<>());
    }})
    .localFilename(new Function<String, String>() {
        @Override
        public String apply(String s) {
            return s.replaceAll(":", "");
        }
    })
    .autoCreateLocalDirectory(true)
    .remoteDirectory(readRemoteDirectory)
    .deleteRemoteFiles(false)
    .preserveTimestamp(true)
    .localDirectory(new File(localDirectory));

and the filter to find the latest file, from its filename:

public class LatestFileOnly implements FileListFilter<ChannelSftp.LsEntry> {

    @Override
    public List<ChannelSftp.LsEntry> filterFiles(ChannelSftp.LsEntry[] files) {
        return Arrays.stream(files)
            .max(byParsedDate)
            .map(Collections::singletonList)
            .orElse(Collections.emptyList());
    }

    public final Comparator<ChannelSftp.LsEntry> byParsedDate = Comparator.comparing(
            (ChannelSftp.LsEntry o) -> {
                        Matcher m = pattern.matcher(o.getFilename());
                        if (m.find()) {
                            return LocalDateTime.parse(m.group(0), DateTimeFormatter.ofPattern("ddMMyyyyHH:mm:ss"));
                        }
                        return LocalDateTime.MIN;
            });
}

Polling after a couple of re-tries: file ignored by the filter

Upvotes: 0

Views: 505

Answers (1)

Artem Bilan
Artem Bilan

Reputation: 121427

as you see now the problem is the LsEntry ends up in the seenSet every time

Yes: that's the problem of the ChannelSftp.LsEntry and AcceptOnceFileListFilter. It doesn't implement a hashCode() method, so every single new instance is different from another even if the target file is the same. For this purpose we have an SftpPersistentAcceptOnceFileListFilter which really compares file names and their getAttrs().getMTime(). See more info in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/file.html#remote-persistent-flf

(monitoring local files to send to another remote location) was somehow triggering AcceptOnceFileListFilter

You said yourself "monitoring", so you probably use over there a Files.inboundAdapter(), which really may be configured with the mentioned AcceptOnceFileListFilter.

Upvotes: 2

Related Questions