CODEWITHSUNDEEP
javaapache-camel
Adrian Jarvis
Adrian Jarvis

Reputation: 66

Apache Camel, resequencer moving files to .camel

I'm trying to resequence the order an application written on top of Apache Camel ingests files. The files need to be processed in a set order and the various aggregations and other processing of the data in the files will fail if the application does no get the files in order. In order to protect the application I am trying to used the resequencer EIP to ensure that the files are processed in order. However I've discovered that the resequencer appears to be causing the files to be moved into the .camel directory before the down stream components are passed the exchange.

I have written a quick example to illustrate the problem:

import java.io.File;
import java.io.IOException;
import org.apache.camel.Exchange;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.apache.commons.io.FileUtils;
import org.junit.Test;

public class CamelFileResequencerTest extends CamelTestSupport {

    @Test
    public void test() throws IOException, InterruptedException {
        File file1 = new File("target/input/1.txt");
        FileUtils.write(file1, "Hello Camel");
        FileUtils.write(new File("target/input/2.txt"), "Hello Camel");
        FileUtils.write(new File("target/input/3.txt"), "Hello Camel");
        Thread.sleep(100l);
        assertTrue(file1.exists());
        FileUtils.write(new File("target/input/4.txt"), "Hello Camel");
        FileUtils.write(new File("target/input/5.txt"), "Hello Camel");
        Thread.sleep(5000l);
        assertTrue(new File("target/output/1.txt").exists());
    }

    @Override
    protected RouteBuilder createRouteBuilder() throws Exception {
        return new RouteBuilder() {

            @Override
            public void configure() throws Exception {
                from("file://target/input")
                    .resequence(header(Exchange.FILE_NAME_ONLY)).batch().size(3)
                    .to("file://target/output/");
            }
        };
    }
}

The test throws exceptions when it tries to write the output file as the input file is no longer in the input directory but has been moved to the /target/input/.camel directory.

Now I can get around this problem by adding processing bean after the resequencer to change the file path to the path in the .camel directory but this seems a bit messy to me. Another option is the resequence the file names rather than the files (i.e. the body of the mesage is the file name as a String and not a Generic file).

Has anyone used the resequencer on files and could offer advice?

Upvotes: 1

Views: 2748

Answers (1)

James Fox
James Fox

Reputation: 691

To stop the file being moved to .camel you can add the noop parameter to the file consumer, in the configure method of the route builder. You can read about this here if using Camel 1.x or here if using Camel 2.0 onwards

from("file://target/input?noop=true")

The reason the files are moved into the .camel directory is not to do with the resequencer but the actual file producer in Camel, as stated in the documentation:

By default, Camel will move consumed files to the sub-folder .camel relative to where the file was consumed.

These files are skipped after this as Camel ignores any files that begin with a dot.

Hope this helps.

Upvotes: 1

Related Questions