CODEWITHSUNDEEP
apache-sparkgoogle-bigquerygoogle-cloud-platformgoogle-cloud-storagegoogle-cloud-dataflow
Munesh
Munesh

Reputation: 1569

Data transfer from Hive to Google Storage/Big Query

I have some Hive tables in an on-premise hadoop cluster.
I need to transfer the tables to BigQuery in google cloud.

Can you suggest any google tools or any open source tools for the data transfer?

Thanks in advance

Upvotes: 0

Views: 1494

Answers (2)

SANN3
SANN3

Reputation: 10069

Using BigQuery migration assessment feature we can migrate from data warehouse to BigQuery. https://cloud.google.com/bigquery/docs/migration-assessment

Upvotes: 0

Dan McGrath
Dan McGrath

Reputation: 42008

BigQuery can import Avro files.

This means you can do something like INSERT overwrite table target_avro_hive_table SELECT * FROM source_hive_table;

You can then load the underlying .avro files into BigQuery via the bq command line tool or using the console UI:

bq load --source_format=AVRO your_dataset.something something.avro

Upvotes: 1

Related Questions