CODEWITHSUNDEEP
redisapache-kafkapublish-subscribe
Sweta Sharma
Sweta Sharma

Reputation: 2754

Difference between Redis and Kafka

Redis can be used as realtime pub-sub just as Kafka.

I am confused which one to use when.

Any use case would be a great help.

Upvotes: 165

Views: 82153

Answers (3)

Behdin Talebi
Behdin Talebi

Reputation: 31

I think the most diffrent between these two service is

  1. Redis supports push-based delivery of messages that means messages published to Redis will be delivered automatically to subscribers immediately but kafka is supports pull-based delivery of messages, meaning that messages published in Kafka are never distributed directly to consumers, consumers subscribe to topics and ask for messages when consumers are ready to deal with them.

  2. Redis does not support the concept of parallelism but Kafka supports parallelism due to the log partitioning of data where multiple consumers consume in consumer groups at the same.

  3. Redis sends the messages to the consumer all at once and later the message is removed. Thus, no one knows where the data is held but kafka is a log, there are always messages; you can monitor this by setting a retention policy for messages. E.g. 7 days retention

  4. redis is used for various use cases such as Session Cache, Full Page Cache (FPC), Leader Boards/Counting, Pub s/ Sub, Queues but kafka has various use cases such as Messaging, Website Activity Tracking, Log Aggregation, Stream Processing, Metrics, Event Sourcing, Commit log.

Upvotes: 3

Nikita Koksharov
Nikita Koksharov

Reputation: 10783

Redis 5.0+ version provides the Stream data structure. It could be considered as a log data structure with delivery guarantees. It offers a set of blocking operations allowing consumers to wait for new data added to a stream by producers, and in addition to that, a concept called Consumer Groups.

Basically Stream structure provides the same capabilities as Kafka.

Here is the documentation https://redis.io/topics/streams-intro

There are two most popular Java clients that support this feature: Redisson and Jedis

Redisson provides ReliableTopic object if reliability of delivery is required. https://github.com/redisson/redisson/wiki/6.-distributed-objects/#613-reliable-topic

Upvotes: 39

Karthikeyan Gopall
Karthikeyan Gopall

Reputation: 5679

Redis pub-sub is mostly like a fire and forget system where all the messages you produced will be delivered to all the consumers at once and the data is kept nowhere. You have limitation in memory with respect to Redis. Also, the number of producers and consumers can affect the performance in Redis.

Kafka, on the other hand, is a high throughput, distributed log that can be used as a queue. Here any number of users can produce and consumers can consume at any time they want. It also provides persistence for the messages sent through the queue.

Final Take:

Use Redis:

  1. If you want a fire and forget kind of system, where all the messages that you produce are delivered instantly to consumers.
  2. If speed is most concerned.
  3. If you can live up with data loss.
  4. If you don't want your system to hold the message that has been sent.
  5. The amount of data that is gonna be dealt with is not huge.

Use kafka:

  1. If you want reliability.
  2. If you want your system to have a copy of messages that has been sent even after consumption.
  3. If you can't live up with data loss.
  4. If Speed is not a big concern.
  5. data size is huge

Upvotes: 230

Related Questions