CODEWITHSUNDEEP
apache-kafkasnowflake-cloud-data-platform
RobMcZag
RobMcZag

Reputation: 657

Moving data from Snowflake to Kafka

The Kafka => Snowflake path is well covered by the Snowflake connector (and other tools). What about moving the data in the other direction?

Has anyone used the generic JDBC source Kafka connector with Snowflake?

The general idea is to have the content of some Snowflake tables made available in Kafka. Kind of a change data capture / snapshot run at some big enough intervals (like one hour / one day).

Any experience to share? Other ideas?

Upvotes: 11

Views: 5120

Answers (1)

Sudhendu
Sudhendu

Reputation: 405

If someone is still looking for an answer, this is possible using Kafka Connect JDBC Source Connector. Here is the step-by-step guide.

Here is the high-level flow:

  • Create a Snowflake Instance.
  • Create a Kafka Environment
  • Setting up Confluent Kafka connect JBDC Source connector.
  • Installing Snowflake JDBC driver.
  • Run the Kafka Connector and do a sanity check.
  • Final step: to see if change data is getting captured!

Upvotes: 7

Related Questions