CODEWITHSUNDEEP
scalaapache-spark
user373201
user373201

Reputation: 11445

Getting a not found error when using split with dataframes

I am working with spark and scala. I have the below code that works fine on spark-shell, but when I try to move it over to Intelij it throws on error, stating unable to find split.

What am I missing, what do I need to import for split to work

var outputDF = inputDF.withColumn(srcColumn, 
split(inputDF.col(srcColumn),splitBy).getItem(selectIndex))

Upvotes: 3

Views: 1891

Answers (1)

Tzach Zohar
Tzach Zohar

Reputation: 37832

You're probably missing this import: 

import org.apache.spark.sql.functions._

(which is provided automatically in a Spark shell)

Upvotes: 8

Related Questions