CODEWITHSUNDEEP
spring-bootapache-kafka
Em Ae
Em Ae

Reputation: 8734

Consuming Kafka Messages in Spring

By following this tutorial, I was able to create a simple producer-consumer example. In my example there was only 1 topic and i was listening to that topic. For that reason, the code in ReceiverConfig makes sense. Specially the point around GROUP_ID_CONFIG i.e., i create topic topic_name and then it was configured in this configuration. Now my question is that, what if I have more than 1 topic. Let's say i have topic_1, topic_2 and so on? Shall i create ReceiverConfig for each individual topic? 

@EnableKafka
@Configuration
public class ReceiverConfig {
    @Value("${spring.kafka.bootstrap-servers}")
    private String bootstrapServers;

    @Bean
    public Map<String, Object> consumerConfigs() {
        Map<String, Object> props = new HashMap<>();
        props.put(BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
        props.put(KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(GROUP_ID_CONFIG, "topic_name");
        props.put(AUTO_OFFSET_RESET_CONFIG, "earliest");

        return props;
    }

    @Bean
    public ConsumerFactory<String, String> consumerFactory() {
        return new DefaultKafkaConsumerFactory<>(consumerConfigs());
    }

    @Bean
    public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<String, String>> kafkaListenerContainerFactory() {
        ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactory());
        return factory;
    }
}

Upvotes: 1

Views: 208

Answers (1)

dbustosp
dbustosp

Reputation: 4478

The short answer is no, you do not need to create multiple configuration for each topic.

Before going any further, I think there is good to specify that the groupId is the group which the Consumer process belongs to and topic to be consumed by the the Consumer process are two different things.

With the sentence below you will tell to the Consumer that its belong to the topic_name group, nothing more. 

props.put(GROUP_ID_CONFIG, "topic_name");

If you want a Consumer to read data from multiple topics, there is a subscribe method which receives a Collections as a parameter, in that way you specify all the topics to read data without having to create a new configuration for each topic.

Please check this example out, you will see the method I mentioned

// Subscribe to the topic.
consumer.subscribe(Collections.singletonList(TOPIC));

Upvotes: 1

Related Questions