CODEWITHSUNDEEP
apache-flink
Tom
Tom

Reputation: 6342

How to run sql-client against yarn-cluster

I am reading https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/sqlClient.html, looks that it illustrates the sql-client functionalities with the standalone cluster.

I would ask whether sql client supports to run against yarn cluster? If yes, I would ask how to do the configuration, I didn't find related how-to on flink.apache.org

Upvotes: 0

Views: 821

Answers (1)

David Anderson
David Anderson

Reputation: 43717

I don't have a yarn cluster available to test this with, but see FLINK-18273 which explains that

The SQL Client YAML has a deployment section. One can use the regular flink run options there and configure e.g. a YARN job session cluster. This is neither tested nor documented but should work due to the architecture.

and also mentions that

The deployment section has also problems with keys that use upper cases. E.g. fromsavepoint != fromSavepoint which requires to use the short option s as a workaround.

Putting those two statements together suggests that putting this entry into sql-env.yaml (xxx is the yarn application id):

deployment:
  yid: xxx

and then starting the client via sql-client embedded -e sql-env.yaml might just work.

See also https://docs.cloudera.com/csa/1.2.0/sql-client/topics/csa-sql-client-session-config.html.

Upvotes: 0

Related Questions