CODEWITHSUNDEEP
scalaapache-spark
Martee
Martee

Reputation: 77

Empty parameter functions passing as arguments

Here are two functions to be used as UDFs: 

def nextString(): String = Random.nextString(10)
def plusOne(a: Int): Int = a + 1

def udfString = udf(nextString)
def udfInt = udf(plusOne)

If I try to use withColumn, myUDF1 will work perfectly fine with udfInt, but throws: can't use Char in Schema for udfString

Probably cause it uses (Int) => (Int) for udfInt type, which is what udf expects

But treats nextString as type String, which obviously leads to an assumption, that I am trying to extract Chars when I apply the function.

It will work if I do something like:

def myUDF: () => String = udf(() => nextString)

Which seems ugly for something that simple. Is there any way to pass udfString as a function, not as String?

Upvotes: 2

Views: 553

Answers (1)

lev
lev

Reputation: 4127

when you write the following code:

def udfString = udf(nextString)

it's the same as writing

val s = nextString
def udfString = udf(s)

this compiles because a string is also a function of Int => Char (see here)

you can tell the compiler that you are passing a function to the udf by:

def udfString = udf(nextString _)

Upvotes: 3

Related Questions