CODEWITHSUNDEEP
hadoophdfsgreenplumexternal-tableshawq
Jones
Jones

Reputation: 399

How to push data from HAWQ into GREENPLUM?

I have this erratic client who wants to push data from HAWQ to GREENPLUM after some pre processing. Is there any way to do this? If not, Is it possible to create an external table in greenplum that reads it from the HDFS in which HAWQ is running?

Any help will be appreciated.

Upvotes: 1

Views: 670

Answers (2)

Gurupreet Singh Bhatia
Gurupreet Singh Bhatia

Reputation: 728

HAWQ is same as Greenplum, only underlying storage is hdfs,

One way is You can create a externale(writable) table in HAWQ which will write your data into a file, now after this you can create a external(readable) table in Greenplum which will read data from that created file

Another way You can copy from one server to another using Standard Input/Output, I use it many times when required to puch data from development environment to Prodcution or vice-versa

Another way You can table a backup using pg_dump/gp_dump for particular table/tables then restore using pg_restore/gp_restore

Thanks

Upvotes: 0

0x0FFF
0x0FFF

Reputation: 5018

The simplest you can do - push the data from HAWQ to HDFS using external writable table and then read it from Greenplum using external readable table using gphdfs protocol. In my opinion this would be the fastest option.

Another option would be to store the data in gzipped CSV files on HDFS and work with them directly from HAWQ. This way when you need this data in Greenplum you can just query it in the same way, as an external table

Upvotes: 3

Related Questions