CODEWITHSUNDEEP
kubernetesapache-flinkflink-streaming
ShemTov
ShemTov

Reputation: 707

flink on kubernetes load external jars to cluster

I am deploying a flink cluster on kubernetes (Session Cluster), and i wanted to know if there is any configuration (on flink-conf.yaml) that i will load external jars to the cluster.

As far i know flink will load all jars under lib/ folder, but lets say i want to upload jar for metrics, and i dont want put it manually in lib folder (becuase every deployment it will delete it), and i dont want make lib/ folder a mount.

The solution i want is to make a /mnt/external-jars library (that will be a mount), and flink will load to it`s classpath all the jars inside this directory.

Is it possible in some way?

Upvotes: 1

Views: 1121

Answers (1)

Till Rohrmann
Till Rohrmann

Reputation: 13346

You could use the config option pipeline.classpaths which takes a semicolon separated list of classpaths. This config option is read when you run the Flink client. The classpaths will be added to the generated Flink JobGraph before it is submitted to the cluster. Note, that the classpaths must be accessible from the cluster nodes.

pipeline.classpaths: file:///mnt/external-jars/MyJar.jar;file:///foobar/other-jars/MyJar2.jar

Upvotes: 3

Related Questions