CODEWITHSUNDEEP
apache-camel
Jimmy
Jimmy

Reputation: 16428

Camel use pollEnrich from same URI multiple times returns null body

I have 2 routes. The first route uses poll enrich to check if a file is present. The second route uses a poll enrich on the same uri to read and process the file. The first route invokes the second via a SEDA queue, like so:

public void configure() throws Exception {
    String myFile = "file://myDir?fileName=MyFile.zip&delete=false&readLock=none";

    from("direct:test")
       .pollEnrich(myFile, 10000)
    .to("seda:myQueue")
    ;

    from("seda:myQueue")
        .pollEnrich(myFile, 10000)
        .log("Do something with the body")
    ;
}

As it stands, if I execute the first route, the poll enrich finds a file, but when the poll enrich in the second route executes, it returns a body of null. If I just execute the second route on its own, it retrieves the file correctly.

Why does the second poll enrich return null, is the file locked? (I was hoping using a combination of noop,readLock, and delete=false would prevent any locking)

Does camel consider the second poll enrich as a duplicate, therefore filtering it out? (I have tried implementing my own IdempotentRepository to return false on contains(), but the second pollEnrich still returns null)

You may wonder why I'm trying to enrich from 2 routes, the first route has to check if a number of files exist, only when all files are present (i.e., pollEnrich doesn't return null) can the second route start processing them.

Is there an alternative to pollEnrich that I can use? I'm thinking that perhaps I'll need to create a bean that retrieves a file by URI and returns it as the body.

I'm using camel 2.11.0

Upvotes: 2

Views: 3447

Answers (5)

Arunas Junevicius
Arunas Junevicius

Reputation: 747

As you've probably learned this does not work as one might expect

noop=true&idempotent=false

my guess is that Camel ignores idempotent=false and as documented uses instance of MemoryMessageIdRepository. To work around this, one can configure file endpoint to use custom idempotent repo:

noop=true&idempotentRepository=#myRepo

and register custom repository in the registry or spring context: 

@Component("myRepo")
public class MyRepo implements IdempotentRepository {
    @Override
    public boolean contains(Object o) {
        return false;
    }
    ...
}

Upvotes: 0

Alex
Alex

Reputation: 31

I realize this is now an old topic, but I just had a similar problem.
I suggest you try the options:

noop=true

which you already have, and

idempotent=false

To tell Camel it is OK to process the same file twice.

Update after testing:

I actually tested this with both settings as suggested above, it works some times, but under moderate load, it fails, i.e. returns null body for some exchanges, although not all.

The documentation indicates that setting noop=true automatically sets idempotent=true, so I am not sure the idempotent setting is being honoured in this case.

Upvotes: 3

Sonu Sabir
Sonu Sabir

Reputation: 3

Try pollEnrich with strategyMethodAllowNull="true". By default , this value is false. When it is false, the aggregation strategy looks for the existing Exchange body, to aggregate the content returned from file. When we make strategyMethodAllowNull="true", the existing body is considered as null. So every time , the content of the file is set into the current exchange body

Upvotes: -1

Ben ODay
Ben ODay

Reputation: 21005

the second route returns NULL because the file was already consumed in the first route...if you are just looking for a signal message when all files are present, then use a file consumer along with an aggregator and possibly a claim check to avoid carrying around large payloads in memory, etc...

Upvotes: 0

hveiga
hveiga

Reputation: 6915

Is there any specific reason why you are not using just one route?

I don't understand why you are using two routes for this. File component can check if the file is there and if it is, pull it. If you are worried about remembering the files so you don't get duplicates, you can use an idempotent repository. At least, based on your question, I don't think you need to complicate the logic using two routes and the content enricher EIP.

Upvotes: 0

Related Questions