CODEWITHSUNDEEP
apache-flink
Dagang Wei
Dagang Wei

Reputation: 26458

Flink on YARN: how do I specify the number of Task Managers

In early versions of Flink, (e.g., 1.6), I can specify the number of Task Managers for both session mode with -n and per-job mode with -yn, but the flags don't exist in later versions of Flink (e.g., 1.12).

Wondering how should I set the number of Task Managers on YARN for newer versions of Flink? Or what are the related properties I can use to control the resources used by Flink?

Upvotes: 0

Views: 918

Answers (1)

David Anderson
David Anderson

Reputation: 43419

In newer versions of Flink, the resource manager dynamically launches task managers as needed to provide the number of slots requested by the job(s) that are submitted. Each task manager will take its configuration either from flink-conf.yaml, or from the parameters provided when the cluster is started via yarn-session.sh.

Upvotes: 3

Related Questions