CODEWITHSUNDEEP
hadoopmapreducehivetez
xxks-kkk
xxks-kkk

Reputation: 2608

"INFO : Tez session hasn't been created yet. Opening session" hang

I'm working with HDP 2.3.0.0-2557 and I'm trying to run a hive query like the following:

beeline> !connect jdbc:hive2://localhost:10000
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: root
Enter password for jdbc:hive2://localhost:10000: ******
Connected to: Apache Hive (version 1.2.1.2.3.0.0-2557)
Driver: Hive JDBC (version 1.2.1.2.3.0.0-2557)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> select * from iitest1;
+----------------+----------------+----------------+----------------+----------------+----------------+-------------------------------------------------------------------------------------------------------+--+
| iitest1.it1c1  | iitest1.it1c2  | iitest1.it1c3  | iitest1.it1c4  | iitest1.it1c5  | iitest1.it1c6   |                                             iitest1.it1c7                                             |
+----------------+----------------+----------------+----------------+----------------+----------------+-------------------------------------------------------------------------------------------------------+--+
| 10             | 84000          | A              | A              | Jonathan       | Liu            | DGGBNEKBRXLBGKTPAGNPEBOQPIPDUAJLTLMRATQPKFPULKKXKGFFWHGOXFFJFLIFKITHKUQSPNFLQKFUIWHXLSTHPBKLMNUSQWOB  |
| 56             | 46000          | F              | F              | 12345          | 12345          | GIIWEVNSPPRLRDKWIXAUFHVNNJISHUWQADQMLWDBRWKWFSXDDXBGDVFRAWEOBTXXRTMMMAOIQUSQBEUGMQKNHFEBFUQECDAOWJMN  |
| 45             | 77000          | K              | K              | Lucy           | King           | HJMTHFFEQBSSCHWSGKNHIQFNKMQFNQHQKNTGSRLVFFJXOAFTDARTMVOCWFIAMICLAVCJBWVKQWAPLBEXMLDGGSVVDJXBDTFLQORQ  |
| NULL           | 77000          | K              | K              | Lucy           | King           | HJMTHFFEQBSSCHWSGKNHIQFNKMQFNQHQKNTGSRLVFFJXOAFTDARTMVOCWFIAMICLAVCJBWVKQWAPLBEXMLDGGSVVDJXBDTFLQORQ  |
+----------------+----------------+----------------+----------------+----------------+----------------+-------------------------------------------------------------------------------------------------------+--+
4 rows selected (8.578 seconds)
0: jdbc:hive2://localhost:10000> select max(it1c1) from iitest1;
INFO  : Tez session hasn't been created yet. Opening session

Aftewards, there is nothing shows up and it seems that something wrong with "Tez"

In addition, I'm trying to run the same query using "mapreduce" instead of "tez" and mapreduce job hangs as well (start hive CLI using hive -hiveconf hive.execution.engine=mr

hive> select max(IT1C1+IT1C2) from iitest1;
Query ID = root_20160713172448_09ccc9e6-c70d-4e56-b428-e511f62db9a6
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1468423564920_0003, Tracking URL = http://sandbox.hortonworks.com:8088/proxy/application_1468423564920_0003/
Kill Command = /usr/hdp/2.3.0.0-2557/hadoop/bin/hadoop job  -kill job_1468423564920_0003
Interrupting... Be patient, this might take some time.
Press Ctrl+C again to kill JVM
killing job with: job_1468423564920_0003
Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0
2016-07-13 17:49:56,510 Stage-1 map = 0%,  reduce = 0%
Ended Job = job_1468423564920_0003 with errors
Error during job, obtaining debugging information...
FAILED: Execution Error, return code 2 from  org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched: 
Stage-Stage-1:  HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
hive> exit
    > ;

I'm wondering if anyone can offer some pointers on how to resolve the issue?

Thanks much!

EDIT 1: I found the following link and based on the link, it seems that there is not enough resources for tez. So, I'm wondering how do I clear up the hogged resources so that I have enough resources for tez.

Upvotes: 1

Views: 9247

Answers (2)

周裕峰
周裕峰

Reputation: 21

I have the same issue with you.

I solved it by modifying the following..

reduce Tez config : tez.am.resource.memory.mb =256

default 2048 .

and Increase Yarn ： Memory allocated for all YARN containers on a node 。

I guess on hive cli use Tez need {tez.am.resource.memory.mb} memory . when {Memory allocated for all YARN containers on a node} no have enough memory . Then Application-id is hang.

Upvotes: 0

Hendrik F
Hendrik F

Reputation: 3930

I've had a similar problem. It might be another process that is blocking your Tez session to run - it was the Spark Thrift Server that was blocking it in my case. Monitor things on your ResourceManager node (Web UI default port is 8088, cf. at bottom of page here). Once the Thrift Servers were killed, it ran like a charm!

Upvotes: 0

Related Questions