CODEWITHSUNDEEP
apache-flinkflink-streamingpyflink
collin power
collin power

Reputation: 1

Access Kafka Metadata in Apache Flink DataStream API

How can i access Kafka metadata when creating source using Apache Flink DataStream Connectors for Kafka ?, i notice in Apache Flink Table API Connector for Kafka, we are able to access the record metadata such as partition and offsets [1], but there is no mention of metadata in DataStream API Connector for Kafka [2].

[1] https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/kafka/#available-metadata

[2] https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/

Right now the only way that i know of to access kafka metadata in Apache Flink is to use Table API to create the source.

Upvotes: 0

Views: 438

Answers (1)

kkrugler
kkrugler

Reputation: 9255

See https://stackoverflow.com/a/74386574/231762 for the answer to a similar question. Short answer, you call .setDeserializer(new KafkaRecordDeserializationSchema(blah)), and your deserializer has access to the Kafka headers.

Upvotes: 0

Related Questions