Clarification required regarding spark scala Array()

Question

Say, I have a dataframe. Few of the column values in that dataframe is Array() i.e., Array[Nothing]. How can i filter these? because if i don't it will throw error if someone tries to print the dataframe or so.

Answer 1

Just filter the empty array.

val df1 = df.withColumn("array", array())
df1.show()

+---+-----+
| id|array|
+---+-----+
|  1|   []|
+---+-----+

val df2 = df1.filter("array != array()")
df2.show()

+---+-----+
| id|array|
+---+-----+
+---+-----+