Reputation: 47
Does data-flow-server and skipper have any active role after the deployment of the stream applications (other than maintaining state). Asked differently, say for example if I have an http source and amqp sink , does any of the traffic from http to amqp go through the data-flow-server or skipper?
Upvotes: 0
Views: 155
Reputation: 5651
Neither SCDF nor Skipper interact with or contribute towards streaming or batch processing. They are both responsible for the designing and the deployment of data pipelines made of streaming/batch applications. To do the designing and the deployment part, we provide a variety of tools, including UI, CLI, RESTful APIs, and Java DSL.
To say this differently, if you deploy a stream or a task data pipeline using SCDF, the applications involved in the data pipeline are solely responsible for the data processing—no data will ever go through SCDF or Skipper. The applications are independently addressing the use-case requirement. SCDF (and Skipper) will help with central monitoring, management, and continuous deliveries of the applications.
You can read more about the responsibilities from the architecture section in docs.
Upvotes: 1