CODEWITHSUNDEEP
scalaapache-sparkintellij-ideacassandra
Hyder Tom
Hyder Tom

Reputation: 373

Scala Code to connect to Spark and Cassandra

I have scala ( IntelliJ) running on my laptop. I also have Spark and Cassandra running on Machine A,B,C ( 3 node Cluster using DataStax, running in Analytics mode). I tried running Scala programs on Cluster, they are running fine. I need to create code and run using IntelliJ on my laptop. How do I connect and run. I know I am making mistake in the code. I used general words. I need to help in writing specific code? Example: Localhost is incorrect. 

 import org.apache.spark.{SparkContext, SparkConf}

  object HelloWorld {
  def main(args: Array[String]) {

    val conf = new SparkConf(true).set("spark:master", "localhost")
    val sc = new SparkContext(conf)
    val data = sc.cassandraTable("my_keyspace", "my_table") 

  }

}

Upvotes: 0

Views: 1605

Answers (1)

undefined_variable
undefined_variable

Reputation: 6228

val conf = new SparkConf().setAppName("APP_NAME")
    .setMaster("local")
    .set("spark.cassandra.connection.host", "localhost")
    .set("spark.cassandra.auth.username", "")
    .set("spark.cassandra.auth.password", "")

Use above code to connect to local spark and cassandra. If your cassandra cluster has authentication enabled then use username and password.

In case you want to connect to remote spark and cassandra cluster then replace localhost with cassandra host and in setMaster use spark:\\SPARK_HOST

Upvotes: 1

Related Questions