CODEWITHSUNDEEP
apache-flinkflink-streamingdatadog
Even
Even

Reputation: 273

Read Flink latency tracking metric in Datadog

I'm following this doc https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/#end-to-end-latency-tracking and enabled metrics.latency.interval in flink-conf.yaml as shown below:

metrics.latency.interval: 60000
metrics.latency.granularity: operator

Now, I have the following questions:

  1. how could I know what kind of metrics(a list of metrics name) are enabled? I didn't find any in metrics UI. enter image description here

  2. Datadog is my reporter, will the latency metrics send to Datadog just like other system metrics listed here https://docs.datadoghq.com/integrations/flink/#data-collected? If yes, what's their name? If no, is there anything I need to do to get them in Datadog?

I'm new to the Flink and the Datadog.Many thanks!

Upvotes: 1

Views: 774

Answers (2)

David Anderson
David Anderson

Reputation: 43707

@monstero has explained where to find the latency metrics -- they are job metrics.

The latency metrics are histogram metrics. latency_p75, for example, is the 75th percentile latency, meaning that 75% of the time the latency was less than the reported value.

In all, you can access the min, max, mean, median, stddev, p75, p90, p95, p98, p99, and p999.

Upvotes: 1

monstereo
monstereo

Reputation: 948

You can access these metrics via rest api integration:

http://{job_manager_address}:8081/jobs/{job_id}/metrics

which will return:

[
  {
    "id": "latency.source_id.3d28eee20f19966ad0843c8183e96045.operator_id.9c9bbdbebfd61a4aaac39e2c417a4f21.operator_subtask_index.7.latency_min"
  },
  {
    "id": "latency.source_id.bca0e5ddee87a6f64a26077804c63e69.operator_id.197249262ed30764bb323b65405e10b4.operator_subtask_index.14.latency_p75"
  },
  {
    "id": "latency.source_id.bca0e5ddee87a6f64a26077804c63e69.operator_id.b9d4ed4c91fec482427d3584100b1c90.operator_subtask_index.12.latency_median"
  },
]

This means that latency from the source_id 3d28eee20... to operator_id 9c9bbdbe with subtask index 7.

However I don't know exact meaning of the latency_p75 or latency_min. Maybe someone else can help us both.

Upvotes: 2

Related Questions