CODEWITHSUNDEEP
hadoop
user3174067
user3174067

Reputation: 1

Accesing remote server get data and put it in hdfs

floks

Here i have a client question. I want to get the tables from sql server(RDBMS) to my hdfs (hadoop cluster). But the servers are in different location.

1)Which is the best way to access the serve,but data is in huge amount.

2)connecting with one sever is okay, we have many servers around the globe we have to get the data from this servers.

3)Can we connect with sqoop remotly to get the data to HDFS.

Upvotes: 0

Views: 1585

Answers (2)

knowone
knowone

Reputation: 840

Before writing down the sqoop import, you need to have a user for each of the remote node which is to be identified by your local DB. For Ex: 

create user 'username'@'<ip of remote node>' IDENTIFIED BY 'password';

You must also have to ensure about the grant permissions to these users depending on your requirement. It's then you can frame the sqoop import, one example as below:

$SQOOP_HOME/bin/sqoop  import  --connect jdbc:mysql://<ip address of remote server node> :port_number/<database_name>  --username user  --password password  --table <table to import>

This question is 5 months old for this answer so I'm hoping the issue would have been resolved but in case someone wanted to go to a step by step procedure for this requirement.

Regards, Adil

Upvotes: 0

Balduz
Balduz

Reputation: 3570

Your question is a little bit unclear, but yes, you can use sqoop to import the data from your servers into HDFS. You need to specify the connection parameters when importing the data:

sqoop import --connect <JDBC connection string> --table <tablename> --username <username> --password <password>

If you need to do multiple imports from multiple servers, I suggest you try Oozie to automate these imports. You can find a tutorial to achieve that here.

Upvotes: 1

Related Questions