CODEWITHSUNDEEP
apache-flinkapache-kafka-connect
user1660472
user1660472

Reputation: 317

Real time Streaming Data Pipeline using Kafka Connect and Flink

I am planning to put together following data pipeline for one of the requirement.
IBM MQ -> Kafka Connect -> Flink -> MongoDB
Flink real time streaming is to perform filtering , applying business rule and enriching incoming records.
IBM MQ part is a legacy component which can not be changed.
Possibly confluent or cloudera platform will be used to house the Kafka and Flink part of the flow.
I could use some thoughts/suggestions around above approach.

Upvotes: 0

Views: 556

Answers (1)

David Anderson
David Anderson

Reputation: 43454

I would take a closer look at whether you really need Kafka Connect. I believe that IBM MQ supports JMS, and there's a JMS compatible connector for Flink in Apache Bahir: http://bahir.apache.org/docs/flink/current/flink-streaming-activemq/.

Upvotes: 1

Related Questions