CODEWITHSUNDEEP
apache-sparkhivehortonworks-data-platformspark-thriftserver
Vittorio Cozzolino
Vittorio Cozzolino

Reputation: 941

Spark Thrift Server - Unable to start (Hive error)

I'm trying to launch the Spark Thrift server (Spark 1.2) on HDP 2.2.0 with Hive 0.14 (latest HortonWorks distribution) but unfortunately I'm getting this error that is quite cryptic for me :

Exception in thread "main" java.lang.RuntimeException: java.lang.NumberFormatException: For input string: "5s"
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)
        at org.apache.spark.sql.hive.HiveContext$$anonfun$4.apply(HiveContext.scala:235)
        at org.apache.spark.sql.hive.HiveContext$$anonfun$4.apply(HiveContext.scala:231)
        at scala.Option.orElse(Option.scala:257)
        at org.apache.spark.sql.hive.HiveContext.x$3$lzycompute(HiveContext.scala:231)
        at org.apache.spark.sql.hive.HiveContext.x$3(HiveContext.scala:229)
        at org.apache.spark.sql.hive.HiveContext.hiveconf$lzycompute(HiveContext.scala:229)
        at org.apache.spark.sql.hive.HiveContext.hiveconf(HiveContext.scala:229)
        at org.apache.spark.sql.hive.HiveContext.runHive(HiveContext.scala:292)
        at org.apache.spark.sql.hive.HiveContext.runSqlHive(HiveContext.scala:276)
        at org.apache.spark.sql.hive.HiveContext.setConf(HiveContext.scala:248)
        at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:91)
        at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:90)
        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
        at org.apache.spark.sql.SQLContext.<init>(SQLContext.scala:90)
        at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:72)
        at org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:51)
        at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:56)
        at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.NumberFormatException: For input string: "5s"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Integer.parseInt(Integer.java:492)
        at java.lang.Integer.parseInt(Integer.java:527)
        at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1094)
        at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1211)
        at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1220)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:58)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)
        ... 26 more

Does anyone can help me fixing this issue? Same happens when I try to execute any command from the Spark shell related to Hive (like creating table, inserting data into existing one etc ..)

Upvotes: 1

Views: 3470

Answers (2)

Harman
Harman

Reputation: 727

You should review your configuration.

  1. Check if you can issue commands from Hive.
  2. If no, look for string "5s" in hive-site.xml

Upvotes: 4

逆袭的龙猫
逆袭的龙猫

Reputation: 103

I have faced the same BUG. HDP2.2.4 have a wrong config file for hive

<property>
  <name>hive.metastore.client.connect.retry.delay</name>
  <value>5s</value>
</property>

should be

<value>5<values>

Hontonworks forum is very slow for user from china. So I can't post this bug to them =_="

Upvotes: 3

Related Questions