CODEWITHSUNDEEP
pythontensorflowgoogle-bigquerygoogle-cloud-dataflow
Alex
Alex

Reputation: 1537

TensorFlow Data Validation and BigQuery

I would like to use TensorFlow Data Validation to analyse and validate data to feed into my ML model. However, my table of raw data is on BigQuery (more than 30gb) and I can't load it as pandas dataframe.

How do I load the data from Bigquery to my noteboow in order to analyse it with TensorFlow Data Validation?

I guess I may use Dataflow, but I can find a proper example

Upvotes: 2

Views: 638

Answers (1)

Filipe Aleixo
Filipe Aleixo

Reputation: 4242

Load everything into a TFRecord file and then pass it to TF Data Validation.

Here you can find an example where they loaded data from BigQuery into TFRecord (they used Apache Beam).

Upvotes: 2

Related Questions