Is there any way to increment specific field by fixed number without writing custom SMTs in Kafka Connect?

Question

Let's assume that I have a table 'student' like below and use Debezium MySQL source connector and JDBC sink connector.

id	class_id	name	age
1	1	'Alice'	15
2	1	'Bob'	16
3	2	'Charlie'	15

When I CDC this table, I just want 'id' and 'class_id' column to be incremented by 1 billion.

The high level architecture diagram to CDC(Change Data Capture) would be like below.

DB1(MySQL) -> MySQL source connector(Debezium) -> Kafka -> (Unwrap SMT) -> (RegexRouter SMT) -> JDBC sink connector -> (?) -> DB2(MySQL)

• Unwrap SMT is for transforming Debezium change event format to be compatible with JDBC sink connector
• RegexRouter SMT is for changing topic name to be the target table I want it to be sync to
• I'm just thinking to place specific SMT(s) to increment 1 billion on specific fields(which is columns) I want.

Would it be any out-of-the-box SMT(s) to be placed on '(?)' part to achieve it, or should I just implement custom SMT?

Answer 1

You could do it in Kafka Streams,. Or use ksqlDB or other stream processing framework. Otherwise, if you need data manipulated before written to Kafka that doesn't match a builtin transform, you'll need your own