CODEWITHSUNDEEP
spring-batchspring-cloudspring-cloud-dataflow
Funsaized
Funsaized

Reputation: 2140

Does spring cloud dataflow for batch use only require skipper?

I'm trying to get a handle on the architecture of spring cloud dataflow. Specifically in using it to orchestrate purely batch pipelines. I've deployed all the components in k8s and noticed I can deploy the skipper component without a middleware (Kafka / RabbitMQ) and run batch jobs successfully through the server component.

Per the docs I can see skipper is used to orchestrate streaming jobs. Is skipper (and subsequently the Kafka/RabbitMQ middleware) a necessary piece of spring cloud dataflow when using for purely batch jobs or tasks?

Upvotes: 1

Views: 503

Answers (1)

Ilayaperumal Gopinathan
Ilayaperumal Gopinathan

Reputation: 4179

Yes, that's correct. Spring Cloud Skipper is not needed when you only run the task (batch) applications.

The idea of Spring Cloud Skipper was to provide continuous deployment experience for Streaming applications. The continuous deployment support for task(batch) applications was embedded into Spring Cloud Data Flow itself since SCDF 2.3.x.

Upvotes: 2

Related Questions