CODEWITHSUNDEEP
apache-camel
Espresso
Espresso

Reputation: 5739

camel: Check if file is "eligible" for processing. If a file is not eligible for procssing move to next

The file process strategy checks databases for specific preprocess conditions/steps were executed before processing a file XYZ.

The files XYZ comes from different sub directories of a root directory that camel scans.

If one of the file is not eligible camel should move to the next one. Not sure what flag should I use if any so it won't bother checking the current file again and again instead of moving to the next one.

My File process Strategy:

 public class MigrFileProcessStrategy<T> extends GenericFileProcessStrategySupport<T> {
private static final Logger log = LoggerFactory.getLogger(MigrFileProcessStrategy.class);

@Autowired
DefaultMigrationProcessor defaultMigrationProcessor;

@Override
public boolean begin(GenericFileOperations<T> operations, GenericFileEndpoint<T> endpoint, Exchange exchange, GenericFile<T> file) throws Exception {
//check if HIST table has entry for this filename, if yes all preprocessing done, file is ready.
boolean readyForProcessing = false;
String fileAbsPath = file.getAbsoluteFilePath();
File inMigrFile = new File(fileAbsPath);
readyForProcessing = defaultMigrationProcessor.isFileReadyForProcessing(inMigrFile);
if (!readyForProcessing) {
    String msg = String.format("\n####Process?:%b  File:%s", readyForProcessing, fileAbsPath);
    log.info(msg);
}
return readyForProcessing;
}

}

My Config:

<bean id="migrFilesToCopy" class="org.apache.camel.component.file.MigrFileFilter">
<!-- Filter for migr files that need to be copied-->
    <property name="caseSensitive" value="true" />
    <property name="excludes" value="**/**/*.migratedLnk, **/_inProgress/**, **/_done/**, **/_failed/**" />
</bean>


    <endpoint id="endpoint_migrFilesToCopy"
        uri="file://#{migrationProcessor.migrRootDir.toFile()}?processStrategy=#migrFileProcessStrategy&amp;directoryMustExist=true&amp;idempotent=true&amp;recursive=true&amp;delete=true&amp;initialDelay=1000&amp;delay=5000&amp;readLock=changed&amp;readLockTimeout=100000&amp;readLockCheckInterval=1000&amp;moveFailed=_failed&amp;maxMessagesPerPoll=10&amp;filter=#migrFilesToCopy" />






    <route id="chMigrate" autoStartup="true">
        <from uri="ref:endpoint_migrFilesToCopy" />
        <pipeline>
            <log message="Procesing file: ${header.CamelFileName}" />
            <!--threads executorServiceRef="migrThreadPool" -->
            <bean ref="migrationProcessor" method="createMetadata" /><!-- MetaDataVo -->
            <bean ref="migrationProcessor" method="createCopyObj" /><!-- CacheCopyObj -->
            <bean ref="migrationProcessor" method="migrate" />
            <!--/threads -->
        </pipeline>
    </route>

Resolved by extending ant filter:

public class MigrFileFilter extends AntPathMatcherGenericFileFilter implements GenericFileFilter { private static final Logger log = LoggerFactory.getLogger(MigrFileFilter.class); @Autowired DefaultMigrationProcessor defaultMigrationProcessor; public MigrFileFilter() { super(); }

@Override
public boolean accept(GenericFile<T> file) {
String fileAbsPath = file.getAbsoluteFilePath();
File inMigrFile = new File(fileAbsPath);
boolean readyForProcessing = false;

if (Files.isDirectory(inMigrFile.toPath())) {
    readyForProcessing = true; //To recursivly process directories.
} else {
    boolean validFilePatten = super.accept(file);
    boolean preprocessDataExist = false;
    if (validFilePatten) {
    preprocessDataExist = defaultMigrationProcessor.isFileReadyForProcessing(inMigrFile);
    }
    readyForProcessing = (validFilePatten && preprocessDataExist);
}
return readyForProcessing;
}

Upvotes: 1

Views: 1986

Answers (1)

Claus Ibsen
Claus Ibsen

Reputation: 55750

Use the filter to filter out unwanted files

You can implement a custom implementation, and just return true|false if you want to include the file or not.

See the section Filter using org.apache.camel.component.file.GenericFileFilter at

Upvotes: 1

Related Questions