CODEWITHSUNDEEP
apache-spark
Raunak Agarwal
Raunak Agarwal

Reputation: 11

Unable to start spark-shell in windows

I'm new to spark and cannot start it in the command prompt by typing spark-shell as it gives me the following lines:

Exception in thread "main" java.lang.ExceptionInInitializerError
        at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
        at org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:611)
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
        at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
        at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:791)
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:761)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:634)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2464)
        at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:222)
        at org.apache.spark.deploy.SparkSubmit$.secMgr$lzycompute$1(SparkSubmit.scala:393)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$secMgr$1(SparkSubmit.scala:393)
        at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:401)
        at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:401)
        at scala.Option.map(Option.scala:146)
        at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:400)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:170)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end 3, length 2
        at java.base/java.lang.String.checkBoundsBeginEnd(Unknown Source)
        at java.base/java.lang.String.substring(Unknown Source)
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:52)
        ... 21 more

Please help.

P.S. Followed this guide https://www.youtube.com/watch?v=WlE7RNdtfwE with the only changes being that I got all the newest versions of the softwares.

Upvotes: 1

Views: 820

Answers (1)

koiralo
koiralo

Reputation: 23099

Try not to use all the latest versions, Use most stable versions.

As the time of this answer, I suggest you use

  1. Java/JDK -> 1.8
  2. Scala -> 2.11.8
  3. Apache Spark -> Spark-2.3.0 Hadoop-2.7

I hope this solves your problem too.

Upvotes: 3

Related Questions