CODEWITHSUNDEEP
google-cloud-dataflow
G B
G B

Reputation: 755

Profiling Google Cloud Dataflow job

What would be the best way to profile a dataflow job if the scale does not permit doing so locally? In the past, I tried using jstack to check what the Java threads are doing on the worker instances, but that doesn't seem to work for anymore. Of course I can use stopwatches and log the measured timing data, but I was hoping maybe there is a better way.

Update: The instructions here still seem to work, with the only difference that instead of installing java with apt-get install openjdk-7-jdk, I had to download it from Oracle's site.

Thanks, GB

Upvotes: 1

Views: 1338

Answers (1)

Ben Chambers
Ben Chambers

Reputation: 6130

As mentioned in the question, you can install jstack if you install the JDK.

We have a Github issue tracking the need for user-code profiling -- check there for progress.

Upvotes: 2

Related Questions