CODEWITHSUNDEEP
google-cloud-platformgoogle-bigquerygoogle-cloud-dataprep
David
David

Reputation: 51

Preprocessing on data stored in BigQuery

I've just started to use GCP and I have some doubts regarding the right use of some of its tools. Particularly, I'm trying to ingest data from Google Analytics into BigQuery. Would it be possible to use Dataprep on data stored in BigQuery? Almost every example I've seen uses Dataprep to visualize data stored in Google Storage, but nothing refers to BigQuery.

Any help would be really appreciated.

Upvotes: 0

Views: 696

Answers (2)

Jofre
Jofre

Reputation: 3898

According to the documentation on Dataprep, you can import BigQuery datasets.

But it might be easier to just to open Dataprep and check the importing options there:

enter image description here

Upvotes: 0

Felipe Hoffa
Felipe Hoffa

Reputation: 59165

You can totally use Dataprep to process data stored in BigQuery. It gives you a great way to visualize how your dataset looks, and interactively define transformations.

Now, do you really want to use Dataprep for this? The transformations will be more expensive and slow, as they will run on Dataflow - which is usually more expensive and slow than doing everything within BigQuery (as the question refers to data that's already in BigQuery).

On the other hand, the interactive environment can help you quickly define what you want and run the created recipe periodically.

See more about this on Lak's "How to schedule a BigQuery ETL job with Dataprep".

Upvotes: 2

Related Questions