CODEWITHSUNDEEP
sqlsql-servert-sqlhadooppolybase
Mani
Mani

Reputation: 344

Connection refused while connecting from polybase to hadoop

While trying to create external table from sql server 2017 to Hadoop in Ubuntu 16.04 it throws the below error

Msg 105019, Level 16, State 1, Line 1 EXTERNAL TABLE access failed due to internal error: 'Java exception raised on call to HdfsBridge_IsDirExist. Java exception message: Call From DESKTOP-VE8KNAG/xxx.xxx.x.xxx to xxx.xxx.x.x:54310 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused: Error [Call From DESKTOP-VE8KNAG/1xxx.xxx.x.xxx to xxx.xxx.x.x:54310 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused] occurred while accessing external file.'

sql query

create EXTERNAL DATA SOURCE [HDP2] 
WITH (TYPE = HADOOP,
 LOCATION = N'hdfs://xxx.xxx.x.x:54310',
 CREDENTIAL = [HDPUser])
GO

create EXTERNAL TABLE [dbo].CLASS_DIM_EXP (
    [CLASS_ID] [varchar](8) NOT NULL,
    [CLASS_DESC] [varchar](100) NULL,
    [INSERT_DATE] [datetime2](7) NOT NULL,
    [LAST_UPDATE_DATETIME] [datetime2](7) NOT NULL)
WITH   (LOCATION='/user/pdw_user',
DATA_SOURCE = HDP2,
FILE_FORMAT = TSV,
REJECT_TYPE = VALUE,
REJECT_VALUE = 0);

Core-site.xml

<property>
  <name>hadoop.tmp.dir</name>
  <value>/app/hadoop/tmp</value>
  <description>A base for other temporary directories.</description>
</property>

<property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:54310</value>
  <description>The name of the default file system.  A URI whose
  scheme and authority determine the FileSystem implementation.  The
  uri's scheme determines the config property (fs.SCHEME.impl) naming
  the FileSystem implementation class.  The uri's authority is used to
  determine the host, port, etc. for a filesystem.</description>
</property>

Is that something needed to be changed?

Upvotes: 1

Views: 701

Answers (1)

Mani
Mani

Reputation: 344

The problem here is cores-site.xml file contains hdfs://localhost:54310 need to replace this with the corresponding IP address hdfs://xxx.xxx.x.x:54310.

Upvotes: 1

Related Questions