CODEWITHSUNDEEP
javaapache-spark
Abigor
Abigor

Reputation: 11

How to convert wrappedArray column in spark dataset to java array?

In my project I got a column after applying collect_list aggregate function, which is of format wrappedArray , I wanted to know is there anyway to convert that wrappedArray to normal array or list So that I can iterate over that column and extract values row by row in that column?

Language:- JAVA Using Apache spark library

By this I am getting that column which is of wrappedArray type

dataset.groupBy( "department","salary","tag")
.agg(collect_list(col("id")));

dataset.foreach(row ->
      row.getAs("id)")

);

I am confused how should I proceed further to iterate in that column.

Upvotes: 1

Views: 939

Answers (2)

Rakesh
Rakesh

Reputation: 31

To convert scala wrappedArray column to Java list in java spark. here line is of type Row.

List<Long> list = scala.collection.JavaConverters.seqAsJavaList(line.getAs("fieldName")));

Upvotes: 0

vinsce
vinsce

Reputation: 1348

If you have a WrappedArray there are several ways to convert it to a Java collection. You can leverage scala.collection.JavaConverters helper methods:

WrappedArray<Long> wrappedArray = row.getAs("ids");
List<Long> list = scala.collection.JavaConverters.seqAsJavaList(wrappedArray.seq());

Alternatively, you can use the Row.getList method to obtain directly a Java List:

List<Long> list = row.getList(row.fieldIndex("ids"));

Upvotes: 1

Related Questions