CODEWITHSUNDEEP
scalaexceptionjdbcapache-sparkhive
chris_chinae
chris_chinae

Reputation: 11

JDBC-HiveServer:'client_protocol is unset!'-Both 1.1.1 in CS

When I ask this question, I have already read many many article through google. Many answers show that is the mismatch version between client-side and server-side. So I decide to copy the jars from server-side to client-side directly, and the result is .... as you know, same exception:

org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default})

It goes well when I connect to hiveserver2 through beeline :) see my connection.

enter image description here

So, I think it will work when I use jdbc too. But, unfortunately, it throws that exception, below is my jars in my project.

    def connect_hive(master:String){
        val conf = new SparkConf()
                        .setMaster(master)
                        .setAppName("Hive")
                        .set("spark.local.dir", "./tmp");
        val sc = new SparkContext(conf);
        val sqlContext = new SQLContext(sc);
        val url = "jdbc:hive2://192.168.40.138:10000";
        val prop= new Properties();
        prop.setProperty("user", "hive");
        prop.setProperty("password", "hive");
        prop.setProperty("driver", "org.apache.hive.jdbc.HiveDriver");
        val conn = DriverManager.getConnection(url, prop);
        sc.stop();
      }

The configment of my server:

  • hadoop 2.7.3
  • spark 1.6.0
  • hive 1.1.1

Does anyone encounter the same situation when connecting hive through spark-JDBC?

Upvotes: 0

Views: 941

Answers (1)

Ram Ghadiyaram
Ram Ghadiyaram

Reputation: 29237

Since beeline works, it is expected that your program also should execute correctly.

print current project class path

you can try some thing like this to understand your self.

import java.net.URL
import java.net.URLClassLoader

import scala.collection.JavaConversions._

object App {

  def main(args: Array[String]) {
    val cl = ClassLoader.getSystemClassLoader
    val urls = cl.asInstanceOf[URLClassLoader].getURLs
    for (url <- urls) {
      println(url.getFile)
    }
  }
}

Also check hive.aux.jars.path=<file urls> to understand what jars are present in the classpath.

Upvotes: 0

Related Questions