CODEWITHSUNDEEP
apache-flink
Francis Nickels
Francis Nickels

Reputation: 149

How can I inspect the internal flink timestamp for an item in a stream using the Processing Time Model?

I am looking to tag the data in my stream with the time it arrived in flink so that I can perform some calculations. I recognize when using the Event Time Model I would have direct control over that, but I was hoping there was some easy way to discover the Timestamp flink was using when making Window decisions on a stream.

Upvotes: 0

Views: 843

Answers (1)

Fabian Hueske
Fabian Hueske

Reputation: 18987

Flink supports three modes to work with time:

  • Processing Time: Events are processed with respect to the current time of each operator
  • Event Time: Events are processes with respect to a timestamp which was manually assigned.
  • Ingestion Time: Events are processed with respect to a timestamp that is automatically assigned when the event is ingested by Flink.

From your description, it seems that you are looking for Ingestion Time. Internally, ingestion time is handed just like event time, but the difference is that the timestamps are automatically assigned (and watermarks are automatically generated).

The time mode is set via the StreamExecutionEnvironment.setStreamTimeCharacteristic() method.

Event time and ingestion time timestamps are not exposed to any function except for the ProcessFunction. A ProcessFunction can read the timestamp of an event in ProcessFunction.processElement() via the Context.timestamp() method.

Upvotes: 3

Related Questions