CODEWITHSUNDEEP
spring-bootapache-kafkamicroservicesspring-kafkachange-data-capture
Marco
Marco

Reputation: 43

Transactional outbox pattern vs ChainedKafkaTransactionManager in Microservices

Using Spring-Kafkas ChainedKafkaTransactionManager I cannot see any point in implementing the transactional outbox pattern in a Spring Boot microservices context.

Putting message producer (i.e. KafkaTemplate's send method) and DB operation in the same transactional block solves exactly the problem that should be solved by the outbox pattern: If any exception is raised in the transactional code neither the db op is commited nor the message is read on the consumer side (configured with read_committed)

This way I dont need an additional table nor any type of CDC code. In summary the Spring Kafka way of transaction synchronization seems much easier to use and implement to me than any implementation of transactional outbox pattern.

Am I missing anything?

public ChainedKafkaTransactionManager chainedTransactionManager(
                        JpaTransactionManager transactionManager,
                        KafkaTransactionManager kafkaTransactionManager) {
        ChainedKafkaTransactionManager chainedKafkaTransactionManager = 
            new ChainedKafkaTransactionManager<>(transactionManager, 
                                                 kafkaTransactionManager);
        
        return chainedKafkaTransactionManager;
    }

    @Bean
    @Primary
    public JpaTransactionManager transactionManager(EntityManagerFactory 
        entityManagerFactory) {
        JpaTransactionManager jpaTransactionManager = 
                    new JpaTransactionManager(entityManagerFactory);
    
        return jpaTransactionManager;
    }

    @Bean
    public KafkaTransactionManager<Object, Object> 
             kafkaTransactionManager(ProducerFactory producerFactory) {
        KafkaTransactionManager kafkaTransactionManager = 
           new KafkaTransactionManager<>(producerFactory);
        
        return kafkaTransactionManager;
    }


    @Transactional(value = "chainedTransactionManager")
    public Customer createCustomer(Customer customer) {
        customer = customerRepository.save(customer);
        kafkaTemplate.send("customer-created-topic","Customer created");
          
        return customer;
    }

Upvotes: 4

Views: 3250

Answers (2)

benaich
benaich

Reputation: 922

you get a weaker guarantee if a data you are trying to update is external to kafka.

Note that exactly-once semantics is guaranteed within the scope of Kafka Streams’ internal processing only; for example, if the event streaming app written in Streams makes an RPC call to update some remote stores, or if it uses a customized client to directly read or write to a Kafka topic, the resulting side effects would not be guaranteed exactly once.

https://www.confluent.fr/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/

Upvotes: 0

Pawel
Pawel

Reputation: 494

I think it doesn't give you the same level of safety. What if something fails between Kafka commit and DB commit.

https://medium.com/dev-genius/transactional-integration-kafka-with-database-7eb5fc270bdc

Upvotes: 2

Related Questions