CODEWITHSUNDEEP
scalaapache-sparkspark-graphx
Akshay Gupta
Akshay Gupta

Reputation: 331

Vertex Property Inheritance - Graphx Scala Spark

--- Edit ---

My main issue is that I do not understand this paragraph given in the Graphx documentation:

In some cases it may be desirable to have vertices with different property types in the same graph. This can be accomplished through inheritance. For example to model users and products as a bipartite graph we might do the following:

class VertexProperty()
case class UserProperty(val name: String) extends VertexProperty
case class ProductProperty(val name: String, val price: Double) extends VertexProperty
// The graph might then have the type:
var graph: Graph[VertexProperty, String] = null

In the above case given RDD's of each UserProperty and ProductProperty and a RDD of EdgeProperty, how does one create a graph of type Graph[VertexProperty, String]. I am looking for an example.

Upvotes: 4

Views: 1318

Answers (3)

hnahak
hnahak

Reputation: 246

This will help you to create a bipartite graph, where vertex property will help you to understand the different class categories.

// High level interface OR VertexProperty

trait Node {   def getVertexID : Long  }

class UserNode(sID: String, sname : String, sAge) extends Node with Serializable { }

class ProductNode(sID: String, sNO : String, sdoe : String) extends Node with Serializable{ }

// Data loading 

val users: RDD[Node]  = sc.textFile("users.txt")
                                 .map { row =>  val cols = row.split(",")
                                         ( new UserNode(cols(0), cols(1), cols(2))
                                  }

val products: RDD[Node]  = sc.textFile("products.txt")
                                 .map { row =>  val cols = row.split(",")
                                        ( new ProductNode(cols(0), cols(1), cols(3)))
                                }

// Join both RDDs 

 val nodes : RDD[Node] = users.++(products)

Upvotes: 2

Akshay Gupta
Akshay Gupta

Reputation: 331

It is a scala question, just convert the the extended type to the abstract type using asInstanceOf, for example:

val variable1: RDD[UserProperty]  = {..your code..}
val variable2: RDD[ProductProperty]  = {..your code..}
val result: RDD[VertexProperty] = SparkContext.union(
variable1.asInstanceOf[VertexProperty],
variable2.asInstanceOf[VertexProperty])

The same goes for edge property, use 

val edge: EdgeProperty = Edge(srcID, dstID, variable.asInstanceOf(EdgeProperty))

Upvotes: 0

Sietse
Sietse

Reputation: 201

You can use a message which can be merged, for example an Iterable[YourClass]. However you have to take into consideration that the size of these kind of merges can become very large.

Upvotes: 0

Related Questions