Ajay K N
Ajay K N

Reputation: 83

Google Bigquery query execution using google cloud dataflow

Is it possible to execute Bigquery's query using Google cloud data flow directly and fetch data, not reading data from table then putting conditions?

For example, PCollections res=p.apply(BigqueryIO.execute("Select col1,col2 from publicdata:samples.shakeseare where ...."))

Instead of reinventing using iterative method what Bigquery queries already implemented, we can use the same directly.

Thanks and Regards

Ajay K N

Upvotes: 3

Views: 709

Answers (1)

Jeremy Lewi
Jeremy Lewi

Reputation: 6766

BigQueryIO currently only supports reading from a Table and not a Query or View (FAQ).

One way to work around this is in your main program to create a BigQuery permanent table by issuing a query before you run your Dataflow job. After, your job runs you could delete the table.

Upvotes: 1

Related Questions