CODEWITHSUNDEEP
apache-kafkaamazon-dynamodbcouchbaseamazon-dynamodb-streamschange-data-capture
Karthik Mannepalli
Karthik Mannepalli

Reputation: 63

CDC (Changed Data Capture) with Couchbase

I have a requirement to capture changes from a stream of data. Below given is my solution.

Data Flows into Kafka -> Consumer Picks up data and inserts/updates (trimmed data) to DynamoDB(We have configured DynamoDB Streams). After every insert/update a stream is generated with changed data, which is then interpreted and processed by a Lambda.

Now my question is if have to replace DynamoDB with Couchbase, will Couchbase provide a CDC out of the box? I am pretty much new to Couchbase and I tried searching for the CDC feature but no direct documentation.

Any pointers would be very helpful! Thanks!

Upvotes: 2

Views: 2118

Answers (2)

Antón R. Yuste
Antón R. Yuste

Reputation: 1010

It seems Change Data Capture (CDC) isn't supported in Couchbase but there are features to notify when documents change. For example, the Source Kafka Connector use that and send documents when changing, including metadata when configured with DefaultSchemaSourceHandler, which should be close enough to CDC:

https://docs.couchbase.com/kafka-connector/current/quickstart.html#defaultschemasourcehandler

Upvotes: 2

Matthew Groves
Matthew Groves

Reputation: 26169

Couchbase has an officially supported Kafka Connector (documentation here).

I'm not familiar with the "CDC" term, but this Couchbase Kafka connector can act as both a sink and a source.. It's not "out of the box" per se, it's a separate connector.

Upvotes: 3

Related Questions