CODEWITHSUNDEEP
debuggingapache-flink
hydronium
hydronium

Reputation: 316

How do I debug Apache Flink?

I've been working on expanding the Apache Flink Python API to better match the Java API and I get weird errors about the types of the data I am working on. Is there a way to attach a Java debugger (e.g. Intellij IDEA) to debug Flink itself?

Upvotes: 6

Views: 7883

Answers (1)

hydronium
hydronium

Reputation: 316

Thanks to Svetlin Zarev for pointing out remote Java debugging. Here are some helpful things I should've found before asking:

  1. Set up remote debugging in IDE/debugger/whatever e.g. for IntelliJ IDEA
  2. Modify Apache Flink JVM arguments by setting env.java.opts in the configuration using the appropriate settings for remote debugging

Or to debug Flink job runners, set up remote debugging and run the job using bin/flink with JVM_ARGS set in your environment. Make sure to set JVM_ARGS only for bin/flink and not for the Flink job manager (i.e. bin/start-local.sh) or the ports will clash.

I've made a little fish shell function that wraps Flink job run so that the proper arguments are set here.

Upvotes: 6

Related Questions