CODEWITHSUNDEEP
netezza
user2509685
user2509685

Reputation: 21

Copy huge data from Netezza to Teradata

We are planning to Copy huge data (250 Billion Rows/5 TB) from Netezza to Teradata. What is the best way to copy this data?

Thanks

Upvotes: 1

Views: 1200

Answers (4)

RDOWE
RDOWE

Reputation: 1

From Netezza to Teradata: 1) Export data off of Netezza: Connect to Netezza using nzodbcsql, it is faster than nzsql.

Invoke NZODBC nzodbcsql -host nzservername -port 5480 -d NZDATABASE -u user -pw password -f /NZ_EXPORT_LOACATION/NZ_EXPORT_SCRIPT

(-f is the NZ_EXPORT_SCRIPT you will call)

The script should look something like this: CREATE EXTERNAL TABLE '/NZ_EXPORT_DIR/DATAFILE.csv' USING (remotesource 'ODBC' DELIMITER ',' NullValue '' escapechar '\' ctrlchars 'yes' crinstring 'yes') AS SELECT * FROM NZDATABASE.TABLE;

If you have a large fact table you can chunk it up by using "where ((1 <= datasliceid) AND (datasliceid <= 31));" put this after your select in the script you'll want to go up to 92 dataslices, or however many you have on your box.

2) Fastload the data into Teradata.

I intially answered the wrong question: but for what it is worth this is how you copy huge data from Teradata to Greenplum:

Fast Export the data from Teradata, you will probably need to run SED on the data that was Fast Exported to remove some of the garbage characteres. SCP/FTP the data over to a GreenPlum ETL head node (GZIP --fast) if you want a smaller file. Create your YAML and GPLoad the data. Start with a small set of data first.

If you want to go from Netezza to Greenplum use the same NZODBC Export to a flat file, then use GPLOAD to load the flat file for the NZODBC Exported flat file.

Upvotes: 0

Alex B
Alex B

Reputation: 2385

You could script it using Python or Perl and run load in parallel. Speed will be limited only by your network bandwidth and Teradata insert.

Upvotes: 0

Niederee
Niederee

Reputation: 4295

Use an External Table to build the flatfile in Netezza.

Then use Teradata MultiLoad or Teradata FastLoad to load the file.

Upvotes: 1

Dan Pichelman
Dan Pichelman

Reputation: 2332

Sneakernet. Seriously. Download the data to microSDs, USB keys, DVDs, whatever. Ship it & reload. It'll be faster than transferring across the Internet.

There's an old quote "Never underestimate the bandwidth of a station wagon full of 9 track tapes". There's an update that computes the bandwidth of a single Boeing 747 full of Blu-Ray disks.
It's impressive.

Upvotes: 0

Related Questions