CODEWITHSUNDEEP
scalacouchbasesql++databricks
Mark
Mark

Reputation: 171

N1QL Query to connect databricks spark 1.6 to couchbase server 4.5

I am trying to setup a connection from Databricks to couchbase server 4.5 and then run a N1QL query.

The scala code below will return 1 record but fails when introducing the N1QL. Any help is appreciated.

    import com.couchbase.client.java.CouchbaseCluster;
    import scala.collection.JavaConversions._;

    import com.couchbase.client.java.query.Select.select;
    import com.couchbase.client.java.query.dsl.Expression;
    import com.couchbase.client.java.query.Query



    // Connect to a cluster on localhost
    val cluster = CouchbaseCluster.create("http://**************")

    // Open the default bucket
    val bucket = cluster.openBucket("travel-sample", "password");

    // Read it back out
    //val streamsense = bucket.get("airline_1004546") - Works and returns one record


  // Create a DataFrame with schema inference
    val ev = sql.read.couchbase(schemaFilter = EqualTo("type", "airline"))

    //Show the inferred schema
     ev.printSchema()

    //query using the data frame
    ev
      .select("id", "type")
      .show(10)


    //issue sql query for the same data (N1ql)
    val query = "SELECT type, meta().id  FROM `travel-sample` LIMIT 10"
    sc
      .couchbaseQuery(N1qlQuery.simple(query))
      .collect()
      .foreach(println)

Upvotes: 1

Views: 378

Answers (2)

1tylermitchell
1tylermitchell

Reputation: 196

In Databricks (and any interactive Spark cloud environment usually) you do not define the cluster nodes, buckets or sc variable, instead you need to set the configuration settings for Spark to use when setting up the Databricks cluster. Use the advanced settings option as shown below.

I've only used this approach with spark2.0 so your mileage may vary.

enter image description here You can remove your cluster and bucket variable initialisation as well.

Upvotes: 2

geraldss
geraldss

Reputation: 2445

You have a syntax error in the N1QL query. You have:

val query = "SELECT type, id  FROM `travel-sample` WHERE LIMIT 10"

You need to either remove the WHERE, or add a condition.

You also need to change id to META().id.

Upvotes: 1

Related Questions