Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible (HIVE)

Question

I'm currently trying to execute only a "SELECT * FROM table" from Hive to ElasticSearch. I'm using cloudera CDH 6.0.1. I've already added the elasticsearch-hadoop-hive-7.1.1 jar to my hive path. I have ElasticSearch 7.1.1 The cloudera stack and elastic running in deferents server but in the same network .

CREATE EXTERNAL TABLE ctrl_rater_resumen_lla_es  
( 
fecha_registro string, 
direccion string, 
linea_b_codigo_prestadora string, 
linea_b_tipo_numero string, 
es_roaming string,
call_duration string, 
linea_b_routing_number string, 
minutos string, fecha string 
) 
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES (
'es.resource' = 'ctrl_rater_resumen_lla/hb',
'es.node' = 'http://10.129.x.xxx',
'es.port' = '9200',
'es.index.auto.create' = 'true',
'es.index.read.missing.as.empty' = 'true',
'es.nodes.discovery'='true',
'es.net.ssl'='false'
'es.nodes.client.only'='false',
'es.nodes.wan.only' = 'true'
'es.net.http.auth.user'='xxxxx',
'es.net.http.auth.pass' = 'xxxxx'
);

The create was successfully

SELECT * FROM ctrl_rater_resumen_lla_es;

Bad status for request TFetchResultsReq(fetchType=0, operationHandle=TOperationHandle(hasResultSet=True, modifiedRowCount=None, operationType=0, operationId=THandleIdentifier(secret='\xbaYG*\xd4wI\xc0\xb8\xf6\x94Q\xa3\xa4IY', guid='\xff\xca\xdb\xb5\x040E\x0e\x8eE\xe4\xf7?t\x1b\x01')), orientation=4, maxRows=100): TFetchResultsResp(status=TStatus(errorCode=0, errorMessage="java.io.IOException: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'", sqlState=None, infoMessages=["*org.apache.hive.service.cli.HiveSQLException:java.io.IOException: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only':25:24", 'org.apache.hive.service.cli.operation.SQLOperation:getNextRowSet:SQLOperation.java:492', 'org.apache.hive.service.cli.operation.OperationManager:getOperationNextRowSet:OperationManager.java:297', 'org.apache.hive.service.cli.session.HiveSessionImpl:fetchResults:HiveSessionImpl.java:852', 'sun.reflect.GeneratedMethodAccessor24:invoke::-1', 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', 'java.lang.reflect.Method:invoke:Method.java:498', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78', 'org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36', 'org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63', 'java.security.AccessController:doPrivileged:AccessController.java:-2', 'javax.security.auth.Subject:doAs:Subject.java:422', 'org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1726', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59', 'com.sun.proxy.$Proxy38:fetchResults::-1', 'org.apache.hive.service.cli.CLIService:fetchResults:CLIService.java:505', 'org.apache.hive.service.cli.thrift.ThriftCLIService:FetchResults:ThriftCLIService.java:702', 'org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults:getResult:TCLIService.java:1717', 'org.apache.hive.service.rpc.thrift.TCLIService$Processor$FetchResults:getResult:TCLIService.java:1702', 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286', 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1149', 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:624', 'java.lang.Thread:run:Thread.java:748', "*java.io.IOException:org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only':29:4", 'org.apache.hadoop.hive.ql.exec.FetchOperator:getNextRow:FetchOperator.java:521', 'org.apache.hadoop.hive.ql.exec.FetchOperator:pushRow:FetchOperator.java:428', 'org.apache.hadoop.hive.ql.exec.FetchTask:fetch:FetchTask.java:146', 'org.apache.hadoop.hive.ql.Driver:getResults:Driver.java:2196', 'org.apache.hive.service.cli.operation.SQLOperation:getNextRowSet:SQLOperation.java:487', "*org.elasticsearch.hadoop.EsHadoopIllegalArgumentException:Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only':35:6", 'org.elasticsearch.hadoop.rest.InitializationUtils:discoverClusterInfo:InitializationUtils.java:340', 'org.elasticsearch.hadoop.hive.HiveUtils:init:HiveUtils.java:197', 'org.elasticsearch.hadoop.hive.EsHiveInputFormat:getSplits:EsHiveInputFormat.java:112', 'org.elasticsearch.hadoop.hive.EsHiveInputFormat:getSplits:EsHiveInputFormat.java:51', 'org.apache.hadoop.hive.ql.exec.FetchOperator:getNextSplits:FetchOperator.java:372', 'org.apache.hadoop.hive.ql.exec.FetchOperator:getRecordReader:FetchOperator.java:304', 'org.apache.hadoop.hive.ql.exec.FetchOperator:getNextRow:FetchOperator.java:459', '*org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException:Connection error (check network and/or proxy settings)- all nodes failed; tried [[localhost:9200]] :41:6', 'org.elasticsearch.hadoop.rest.NetworkClient:execute:NetworkClient.java:152', 'org.elasticsearch.hadoop.rest.RestClient:execute:RestClient.java:424', 'org.elasticsearch.hadoop.rest.RestClient:execute:RestClient.java:388', 'org.elasticsearch.hadoop.rest.RestClient:execute:RestClient.java:392', 'org.elasticsearch.hadoop.rest.RestClient:get:RestClient.java:168', 'org.elasticsearch.hadoop.rest.RestClient:mainInfo:RestClient.java:735', 'org.elasticsearch.hadoop.rest.InitializationUtils:discoverClusterInfo:InitializationUtils.java:330'], statusCode=3), results=None, hasMoreRows=None)

Answer 1

The correct property is "es.nodes", not "es.node". The default is "localhost", so you are trying to connect to your localhost instead of the node you want to connect to. See the documentation for more information.

You might also want to think about whether you need the property "es.nodes.wan.only" to be set to true if you are not connecting to a cloude environment, as that will disable the automatic discovery of other nodes in your network as is explained in the documentation (you need to scroll down a bit). It will force the system to use the "es.nodes" property, which per default tries to connect to localhost. That's why you are getting the error, but even if you make it work that setting will have an impact on your performance (original emphasis):

Note that in this mode, performance is highly affected.