CODEWITHSUNDEEP
sqlscalaapache-spark
Pingjiang Li
Pingjiang Li

Reputation: 747

what is the difference between array<string> and Array<String> in Spark?

I used this code to aggragate the grouped data:

val result=union_df.orderBy(desc("timestamp")).groupBy("id").agg(collect_set("region") as "region")

Then I got the datatype:

org.apache.spark.sql.DataFrame = [id: string, region: array<string>]

What is the different between array<string> and Array<String>? How do I iterate over array<string> in map function (there is no getArray function for Row)?

Upvotes: 0

Views: 168

Answers (1)

koiralo
koiralo

Reputation: 23109

There is a getSeq() function that returns a array or you can use getAs method by getAs[Array[String]]() or getAs[Seq[String]]()

array<string> is scala.collection.mutable.WrappedArray[String]] which is same as a Array<String> with a wrapper on it which is used inside a Row.

Upvotes: 2

Related Questions