CODEWITHSUNDEEP
postgresqldebezium
user6702954
user6702954

Reputation: 85

postgres debezium returning different dates format

I got a postgres debezium connector running and I can see even the data on database looks like :

effectivedates_effectivestartdate | date
 
effectivedates_effectiveenddate 
---------------------------------+--------
 9999-12-31

when I check the CDC topics the data appears to be like :

}, "effectivedates_effectiveenddate": { "string": "10000-12-31" }

and data is landing on S3 buckect like :

max(after.effectivedates_effectiveenddate AS `effectivedates_effectiveenddate`)|
+-------------------------------------------------------------------------------+
|10000-12-31                                                                    |
+-------------------------------------------------------------------------------+

so I wonder if anyone knows that is causing this behaviour. Is there a configuration parameter that might help to fix this. thanks.

Upvotes: 0

Views: 433

Answers (1)

prashant.kr.mod
prashant.kr.mod

Reputation: 1702

Do the date transformations:

    "transforms": "route,unwrap,DATEPARTITION",

    "transforms.route.type": "org.apache.kafka.connect.transforms.RegexRouter",
    "transforms.route.regex": "([^.]+)\\.([^.]+)\\.([^.]+)",
    "transforms.route.replacement": "$3",

    "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
    "transforms.DATEPARTITION.type":"org.apache.kafka.connect.transforms.TimestampConverter$Value",
    "transforms.DATEPARTITION.target.type":"string",
    "transforms.DATEPARTITION.field":"DATEPARTITION",
    "transforms.DATEPARTITION.format":"dd-MM-yyyy" ,
    "time.precision.mode":"connect",

Upvotes: 0

Related Questions