CODEWITHSUNDEEP
scalaapache-sparkdatabricksgraphframes
rubik90
rubik90

Reputation: 67

Graphframe error in Scala/Spark

I wrote this code lines in Scala 2.11 into Databricks:

import org.graphframes._

val user_ridotto = sqlContext.sql("SELECT * FROM userRidotto")

var users_1 = user_ridotto.select("user_id", "name", "city", "num_fr",  
"fans", "review_count", "importance").withColumnRenamed("user_id", "id")
val users = users_1.withColumn("ridotto", lit("ridotto"))
var edges_1 = user_ridotto.select($"user_id" as "src", explode($"friends") 
as 
"dst", lit(1))
val graph_1 = GraphFrame(users, edges_1)
println("users: " + users.count + ", archi_1: " + edges_1.count)

val paths = graph_1.find("(a)-[e]->(b)")
.filter("a.ridotto='ridotto'")
.filter("b.ridotto='ridotto'")
val edges = paths.select("e.src", "e.dst")
val graph = GraphFrame(graph_1.vertices, edges)
println("users: " + users.count + ", archi: " + edges.count)

but it returns me these error:

notebook:1: error: object graphframes is not a member of package org
import org.graphframes._
           ^
notebook:8: error: not found: value GraphFrame
val graph_1 = GraphFrame(users, edges_1)
          ^
notebook:15: error: not found: value GraphFrame
val graph = GraphFrame(graph_1.vertices, edges)

Thanks in advance Sincerely Antonio

Upvotes: 1

Views: 2491

Answers (1)

Karl Dailey
Karl Dailey

Reputation: 118

the library isn't loded naitively, so you need to import the jar file under libraries. you can import the file under workspace (there should be a message box that allows you to import jar files specifically) and then attach it. after attaching you need to restart the cluster

you can get the jar file at: graphframes

Upvotes: 1

Related Questions