CODEWITHSUNDEEP
apache-camel
dustmachine
dustmachine

Reputation: 10824

How to chain two separate camel routes together for throttling

I have two separate routes running in an application and I want to control the total amount of work in flight across the entire path.

Route 1:  Gzipped file on SFTP --> unzip --> local directory
Route 2:  local directory --> process stuff --> Kafka

If route 2 has problems or falls behind in its work, I don't want route 1 to fill up the local directory. How can I limit the total number of files sitting in local dir waiting to be processed?

(if it was a single route I may be able to throttle() easier, but are there other options to look at the overall picture of multiple routes?)

Upvotes: 4

Views: 743

Answers (1)

Claus Ibsen
Claus Ibsen

Reputation: 55540

You can implement a custom RoutePolicy where you check the number of files in that directory and if its bigger than X then suspend the route, and resume it again if its lower than X.

See more details at the Camel docs: http://camel.apache.org/routepolicy.html

You can look at the existing ThrottlingInflightRoutePolicy how its implemented for inspiration.

Upvotes: 5

Related Questions