CODEWITHSUNDEEP
springspring-integration
mike.mike
mike.mike

Reputation: 21

File inbound channel adapter process same file concurrently

I have a requirement to process any file that is put in a specific folder. Those files can have same names. Later it will be processed by a proprietary bean that will rename the file extension based on the duplicate status of the filename (xxx.origin if the file has never been processed before or xxx.duplicate for the opposite condition).

Based on this requirement, I decided to use file-inbound-channel-adapter, with spring-integration-3.0.8. I set the prevent-duplicates attribute to false, so I can process all files. However, I found out that 1 file can being processed by 2 file-inbound-channel-adapter concurrently (even though the second adapter was created by accident - initialized twice in context and servlet). But this is a problem for me, because I actually intended to create more than 1 file-inbound-channel-adapter to process all files in the same folder and I get a result that 1 input file is being processed twice by 2 beans at the same time which got me 2 output file (xxx.origin and xxx.duplicate). I can't use FileLocker also, because the file-to-string-transformer can't work due to the lock status of the file.

Is there any way I can achieve the requirement?

Upvotes: 1

Views: 924

Answers (1)

Gary Russell
Gary Russell

Reputation: 174779

It's not clear why you need multiple adapters polling the same directory - if you need to achieve concurrent processing, use a single adapter and add a task executor to the poller (or use an executor channel downstream).

If you must have multiple adapter instances, and you can use a newer version of Spring Integration (4.1.7 or higher, current version is 4.2.6) you can use a a shared AcceptOnceFileListFilter which is now a ResettableFileListFilter so when you rename the file you can remove it from the filter.

If you can't go to a newer version of Spring Integration, you will need a custom FileListFilter that allows you to remove the file after processing.

Upvotes: 1

Related Questions