CODEWITHSUNDEEP
spring-bootapache-kafkaspring-batch
Ash3060
Ash3060

Reputation: 340

Spring Batch and Apache Kafka

I am in learning phase of Kafka.I came across this video This Video

This confuses me alot. I am able to understand kafka consumer and producer and i can see lot of reference materials related to that. We have batch listeners already there so why we need spring batch support here .Is there any specific advantage of using spring kafka batch over using normal batch listeners? Please help me in understanding as i can't see any reference materials comparing both.What i felt that we have more freedom and customisations using normal consumer and producer.Please correct me if i am wrong.

Upvotes: 0

Views: 3551

Answers (1)

Mahmoud Ben Hassine
Mahmoud Ben Hassine

Reputation: 31600

Spring Batch is a batch processing framework (fixed data sets) while Kafka is a streaming platform (infinite data streams). Those tools address two different types of requirements and use cases.

However, there are many cases where you want to have a "bridge" between these two worlds. Here are a couple examples:

  • Replay a stream of events to create an application state up to a certain time: Here you can use a Spring Batch job that reads a Kafka topic from the beginning and replays all events (The KafkaItemReader can be helpful here)
  • Inject a set of events from a file or a database table in a live stream. The KafkaItemWriter can be used in this case.
  • etc

The advantage of using a Spring Batch job over a regular batch listener is all what Spring Batch offers in terms of transaction management, state management for restartability, fault-tolerance features, etc.

Upvotes: 4

Related Questions