CODEWITHSUNDEEP
sql-serverhadooppolybase
Paniz Asghari
Paniz Asghari

Reputation: 315

Hadopp Polybase: the query processor encountered an unexpected error during the processing of a remote query phase

I am new to Polybase, but I have been able to create an external data source and external table from SQL Server polybase to Hadoop HortonWorks 3.1 with Kerberos and HA enabled on.

When I try to have a select query from my external table the below error is thrown:

Internal Query Processor Error: The query processor encountered an unexpected error during the processing of a remote query phase.

I have already read the documents and possible reasons which states that I need to restart my Polybase services after installation but I have already restarted all SQL server related services plus polybase ones. Also, my polybase Data Movement service sometimes stops by itself; however while it is running, I face the same error.

Here is my external data source

CREATE EXTERNAL DATA SOURCE [Hive_test3] 
WITH 
(TYPE = HADOOP,
LOCATION = N'hdfs://xxxxxxxxxx:8020', 
RESOURCE_MANAGER_LOCATION = N'xxxxxxxxx:8032',
CREDENTIAL = [HadoopUser2])

and my external table as:

CREATE EXTERNAL TABLE [dbo].[municipal_test]
(
    [id] [int] NULL,
    [city_id] [int] NULL,
    [name] [varchar](250) NULL
)
WITH (DATA_SOURCE = [Hive_test3],
LOCATION = N'/user/xxxx/xxxx.csv',
FILE_FORMAT = [csvFileFormat],
REJECT_TYPE = VALUE,REJECT_VALUE = 500)

and even the following query fails:

SELECT 1
FROM 
[dbo].[municipal_test]

Can you please kindly help me out?

Upvotes: 0

Views: 866

Answers (2)

Paniz Asghari
Paniz Asghari

Reputation: 315

The issue was that we had installed SQL and Polybase services before we migrate to another domain. Although, after change of domain we had changed the service accounts too; however, the logs showed that one or two services were still using the old ones. Hence, uninstall and reinstallation using the domain accounts solved the issue.

Upvotes: 1

pabechevb
pabechevb

Reputation: 131

Please check you've done the following:

  1. Configured 'polybase enabled'
  2. Configured 'hadoop connectivity'
  3. Modified 'yarn-site.xml'
  4. Created the master key encryption
  5. In Hadoop you've created the user at the OS level and granted folder permissions to it

Also, please provide the code for your external file format (it was not included when I wrote this answer). Here is additional information regarding Kerberos: https://learn.microsoft.com/en-us/sql/relational-databases/polybase/polybase-configuration, and here is additional information for troubleshooting: https://learn.microsoft.com/en-us/sql/relational-databases/polybase/polybase-troubleshoot-connectivity.

Upvotes: 1

Related Questions