CODEWITHSUNDEEP
scalaapache-spark
Metadata
Metadata

Reputation: 2085

How to convert a row from Dataframe to String

I have a dataframe which contains only one row with the column name: source_column in the below format:

forecast_id:bigInt|period:numeric|name:char(50)|location:char(50)

I want to retrieve this value into a String and then split it on the regex | First I tried converting the row from the DataFrame into the String by following way so that I can check if the row is converted to String:

val sourceColDataTypes = sourceCols.select("source_columns").rdd.map(x => x.toString()).collect()

When I try to print: println(sourceColDataTypes) to check the content, I see [Ljava.lang.String;@19bbb216 I couldn't understand the mistake here. Could anyone let me know how can I properly fetch a row from a dataframe and convert it to String.

Upvotes: 0

Views: 3564

Answers (1)

1pluszara
1pluszara

Reputation: 1528

You can also try this:

df.show()

//Input data
//+-----------+----------+--------+--------+
//|forecast_id|period    |name    |location|
//+-----------+----------+--------+--------+
//|1000       |period1000|name1000|loc1000 |
//+-----------+----------+--------+--------+

df.map(_.mkString(",")).show(false)

//Output:
//+--------------------------------+
//|value                           |
//+--------------------------------+
//|1000,period1000,name1000,loc1000|
//+--------------------------------+        

df.rdd.map(_.mkString(",")).collect.foreach(println)

//1000,period1000,name1000,loc1000

Upvotes: 2

Related Questions