CODEWITHSUNDEEP
apache-kafka
Carlos Barcellos
Carlos Barcellos

Reputation: 572

In Kafka, how producers receive response from consumers?

In a very simple use case in old client/server model, for requesting some info from a database server, I would create a connection, send the request (query) and receive the response.

How is the Kafka way of doing such a request?

For instance, let's suppose I have a customer ID and want to retrieve the user info that's stored somewhere in the Kafka infrastructure, do I have to publish in one topic and consume the info from another topic (act as producer/consumer)?

Is this the supposed way of using Kafka?

EDIT 1:

In order to elaborate what I'm trying to achieve I'm going to try to clarify the operation:

1 - Some producer insert a costumer in the Kafka "customers" topic. 2 - Some consumer reads this topic, connect to a database and insert it the database. 3 - Due to business constraints, the Kafka logs are kept only for a month. 4 - After a month, some client attached to Kafka infrastructure wants to retrieve the information of a specific customer that are located in the database (it does not even know about a database).

Would it make sense for this client to publish a message in a specific topic, let's say "i-want-a-costumer-info" and then waits for a response in a topic "customers-info" filtering by the customer ID? In this way, the client I mention in (4) would be acting both as a producer and a consumer.

Upvotes: 1

Views: 1096

Answers (1)

Carlos Barcellos
Carlos Barcellos

Reputation: 572

The Kafka way of such achievement is using what is very well documented in the Kafka Streams interactive queries, please refer to Kafka Streams Interactive Queries.

Basically, we have to follow these steps:

To query the full state of your application, you must connect the various fragments of the state, including:

  • query local state stores
  • discover all running instances of your application in the network and their state stores
  • communicate with these instances over the network (e.g., an RPC layer)

Connecting these fragments enables communication between instances of the same app and communication from other applications for Interactive Queries

Upvotes: 1

Related Questions