CODEWITHSUNDEEP
scalaapache-spark
Deepak Garg
Deepak Garg

Reputation: 159

How we see different implementation of same function

I want to understand how different implementation of same method is done in Spark Scala.

def createDataFrame[A <: Product](data: Seq[A])(implicit evidence$3: reflect.runtime.universe.TypeTag[A]): org.apache.spark.sql.DataFrame
def createDataFrame(rdd: org.apache.spark.api.java.JavaRDD[_],beanClass: Class[_]): org.apache.spark.sql.DataFrame
def createDataFrame(rowRDD: org.apache.spark.api.java.JavaRDD[org.apache.spark.sql.Row],schema: org.apache.spark.sql.types.StructType): org.apache.spark.sql.DataFr
ame
def createDataFrame(data: java.util.List[_],beanClass: Class[_]): org.apache.spark.sql.DataFrame
def createDataFrame(rows: java.util.List[org.apache.spark.sql.Row],schema: org.apache.spark.sql.types.StructType): org.apache.spark.sql.DataFrame
def createDataFrame(rdd: org.apache.spark.rdd.RDD[_],beanClass: Class[_]): org.apache.spark.sql.DataFrame
def createDataFrame[A <: Product](rdd: org.apache.spark.rdd.RDD[A])(implicit evidence$2: reflect.runtime.universe.TypeTag[A]): org.apache.spark.sql.DataFrame
def createDataFrame(rowRDD: org.apache.spark.rdd.RDD[org.apache.spark.sql.Row],schema: org.apache.spark.sql.types.StructType): org.apache.spark.sql.DataFrame

Here is the different implementation of createDataFrame method? How same method createDataFrame can take different parametets yet output is same - Dataframe?

Upvotes: 0

Views: 135

Answers (1)

Suhas NM
Suhas NM

Reputation: 1080

This is a feature of Scala language, similar features are available in many other languages like java, C++ etc.

This feature is called method overloading, it works like this:

def myMethod(a:Int,b:Int) = a + b

def myMethod(a:String,B:String) = a + " " + b  

myMethod(1,2) //-----> returns 3, first method is called

myMethod("Hello","world")  //-----> returns "Hello World", second method is called

General rules of method overloading:

  • Names of all methods should be the same
  • Number of parameters/type of parameters or both has to be unique across all methods with same name

Here createDataFrame function is overloaded many times to accommodate different scenarios that are used to create a dataframe, to support multiple sources. If you closely look into the source code, you will find that none of those methods which you have highlighted have the same method signature.

Upvotes: 3

Related Questions