CODEWITHSUNDEEP
google-bigquerygoogle-cloud-dataflowapache-beam-io
Graham Polley
Graham Polley

Reputation: 14781

Dataflow pipeline details for BigQuery source/sinks not displaying

According to this announcement by the Google Dataflow team, we should be able to see the details of our BigQuery sources and sinks in the console if we use the 1.6 SDK.

However, although the new "Pipeline Options" do indeed show up, the details of the BigQuery source/sinks do not display. Our pipeline is running in Batch mode. Job id for reference is: 2016-06-23_04_38_00-7749718775594891108

How do we get the details of the BigQuery source/sinks to show up?

enter image description here

enter image description here

enter image description here

Upvotes: 0

Views: 135

Answers (1)

Alex Amato
Alex Amato

Reputation: 1725

I'm one of the dataflow developers who added this feature to the dataflow UI.

Currently these details are only visible when you click on the expanded steps. If click the arrow icon to fully expand your sources/sinks and click on the steps that show up after its expanded then you can see details that were specified for that step such as the BigQuery Query or table information.

Is there a particular detail/parameter you are looking for and not seeing for a step in your pipeline?

Upvotes: 1

Related Questions