CODEWITHSUNDEEP
performancejdbcdb2
soulfly1983
soulfly1983

Reputation: 177

How to Move large amount of data between Databases?

I need to compare data from two databases (both of them are DB2) which are on different servers with no existing connection between them. Because both of the db's are used in production I don't want to overload them, therefore I will create a new db (probably MySQL) on my local machine, extract data from both of the DB2's, insert into MySQL and do the comparison locally.

I would like to do this in Java, so my question is how to do this task as effectively as possible, without overloading the production databases. I've done some research and came up with the following points:

Is there anything else that I can do, or any other suggestions?

Thank you

Upvotes: 0

Views: 470

Answers (2)

Fred Sobotka
Fred Sobotka

Reputation: 5332

DB2 for Linux UNIX, and Windows includes the EXPORT utility as part of its runtime client. This utility can be pointed at a DB2 database on z/OS to quickly drain a table (or query result set) into a flatfile on your client machine. You can choose whether the flatfile is delimited, fixed width, or DB2's proprietary IXF format. Your z/OS DBA should be able to help you configure the client on your workstation and bind the necessary packages into the z/OS databases as required by the EXPORT utility.

Once you have the flatfiles on your client, you can compare them however you like.

Upvotes: 2

Chris Gerken
Chris Gerken

Reputation: 16392

Sounds like a great job for map reduce (hadoop). One job with two mappers, one for each DB and a reducer to do the compares. It can scale to as many processors as you need, or just run on a single machine.

Upvotes: 0

Related Questions